Duplicate detection - step 2: review and decide on candidate pairs¶
This notebook runs the second part of the duplicate detection algorithm on a dataframe with the following columns:
archiveType(used for duplicate detection algorithm)dataSetNamedatasetIdgeo_meanElev(used for duplicate detection algorithm)geo_meanLat(used for duplicate detection algorithm)geo_meanLon(used for duplicate detection algorithm)geo_siteName(used for duplicate detection algorithm)interpretation_directioninterpretation_seasonalityinterpretation_variableinterpretation_variableDetailsoriginalDataURLoriginalDatabasepaleoData_notespaleoData_proxy(used for duplicate detection algorithm)paleoData_unitspaleoData_values(used for duplicate detection algorithm, test for correlation, RMSE, correlation of 1st difference, RMSE of 1st difference)paleoData_variableNameyear(used for duplicate detection algorithm)yearUnits
This interactive notebook runs a duplicate decision algorithm for a specific database, following the identification of the potential duplicate candidate pairs. The algorithm walks the operator through each of the detected duplicate candidate pairs from dup_detection.ipynb and runs a decision process to decide whether to keep or reject the identified records.
The confirmed 'true' duplicates are saved in
data/DATABASENAME/duplicate_detection/duplicate_decisions_DATABASENAME_AUTHORINITIALS_YY-MM-DD.csv
10/11/2025 LL: tidied up with revised data organisation and prepared for documentation 27/11/2024 LL: Changed hierarchy FE23>PAGES 2k 22/10/2024 v1: Updated the decision process: - created backup decision file which is intermediately saved - outputs URL which can be copied and pasted into browser - implemented a composite option in the decision process, to create a composite of two records
Author: Lucie Luecke, created 27/9/2024
Note: The algorithm can be either started from scratch or from a backup file:
Intialisation¶
Set up working environment¶
Make sure the repo_root is added correctly, it should be: your_root_dir/dod2k This should be the working directory throughout this notebook (and all other notebooks).
In [11]:
Copied!
%load_ext autoreload
%autoreload 2
import sys
import os
from pathlib import Path
# Add parent directory to path (works from any notebook in notebooks/)
# the repo_root should be the parent directory of the notebooks folder
current_dir = Path().resolve()
# Determine repo root
if current_dir.name == 'dod2k': repo_root = current_dir
elif current_dir.parent.name == 'dod2k': repo_root = current_dir.parent
else: raise Exception('Please review the repo root structure (see first cell).')
# Update cwd and path only if needed
if os.getcwd() != str(repo_root):
os.chdir(repo_root)
if str(repo_root) not in sys.path:
sys.path.insert(0, str(repo_root))
print(f"Repo root: {repo_root}")
if str(os.getcwd())==str(repo_root):
print(f"Working directory matches repo root. ")
%load_ext autoreload
%autoreload 2
import sys
import os
from pathlib import Path
# Add parent directory to path (works from any notebook in notebooks/)
# the repo_root should be the parent directory of the notebooks folder
current_dir = Path().resolve()
# Determine repo root
if current_dir.name == 'dod2k': repo_root = current_dir
elif current_dir.parent.name == 'dod2k': repo_root = current_dir.parent
else: raise Exception('Please review the repo root structure (see first cell).')
# Update cwd and path only if needed
if os.getcwd() != str(repo_root):
os.chdir(repo_root)
if str(repo_root) not in sys.path:
sys.path.insert(0, str(repo_root))
print(f"Repo root: {repo_root}")
if str(os.getcwd())==str(repo_root):
print(f"Working directory matches repo root. ")
The autoreload extension is already loaded. To reload it, use: %reload_ext autoreload Repo root: /home/jupyter-lluecke/dod2k_v2.0/dod2k Working directory matches repo root.
In [12]:
Copied!
import pandas as pd
import numpy as np
import datetime
from dod2k_utilities import ut_functions as utf # contains utility functions
from dod2k_utilities import ut_duplicate_search as dup # contains utility functions
import pandas as pd
import numpy as np
import datetime
from dod2k_utilities import ut_functions as utf # contains utility functions
from dod2k_utilities import ut_duplicate_search as dup # contains utility functions
Load dataset¶
Define the dataset which needs to be screened for duplicates. Input files for the duplicate detection mechanism need to be compact dataframes (pandas dataframes with standardised columns and entry formatting).
The function load_compact_dataframe_from_csv loads the dataframe from a csv file from data\DB\, with DB the name of the database. The database name (db_name) can be
pages2kch2kiso2ksisalfe23
for the individual databases,
or use
all_merged
to load the merged database of all individual databases, or can be any user defined compact dataframe.
In [13]:
Copied!
# load dataframe
db_name='all_merged'
# db_name='dup_test'
df = utf.load_compact_dataframe_from_csv(db_name)
print(df.info())
df.name = db_name
# load dataframe
db_name='all_merged'
# db_name='dup_test'
df = utf.load_compact_dataframe_from_csv(db_name)
print(df.info())
df.name = db_name
<class 'pandas.core.frame.DataFrame'> RangeIndex: 5320 entries, 0 to 5319 Data columns (total 21 columns): # Column Non-Null Count Dtype --- ------ -------------- ----- 0 archiveType 5320 non-null object 1 dataSetName 5320 non-null object 2 datasetId 5320 non-null object 3 geo_meanElev 5221 non-null float32 4 geo_meanLat 5320 non-null float32 5 geo_meanLon 5320 non-null float32 6 geo_siteName 5320 non-null object 7 interpretation_direction 5320 non-null object 8 interpretation_seasonality 5320 non-null object 9 interpretation_variable 5320 non-null object 10 interpretation_variableDetail 5320 non-null object 11 originalDataURL 5320 non-null object 12 originalDatabase 5320 non-null object 13 paleoData_notes 5320 non-null object 14 paleoData_proxy 5320 non-null object 15 paleoData_sensorSpecies 5320 non-null object 16 paleoData_units 5320 non-null object 17 paleoData_values 5320 non-null object 18 paleoData_variableName 5320 non-null object 19 year 5320 non-null object 20 yearUnits 5320 non-null object dtypes: float32(3), object(18) memory usage: 810.6+ KB None
Input operator's credentials¶
In order to keep maximum transparency and reproduceability, put in the operator's credentials here.
These details are used to flag the intermediate output files and provided along with the final duplicate free dataset.
In [14]:
Copied!
initials = 'LL'
fullname = 'Lucie Luecke'
email = 'ljluec1@st-andrews.ac.uk'
# initials = 'MNE'
# fullname = 'Michael Evans'
# email = 'mnevans@umd.edu'
operator_details = [initials, fullname, email]
initials = 'LL'
fullname = 'Lucie Luecke'
email = 'ljluec1@st-andrews.ac.uk'
# initials = 'MNE'
# fullname = 'Michael Evans'
# email = 'mnevans@umd.edu'
operator_details = [initials, fullname, email]
Duplicate decision process¶
We now start the duplicate decision process.
Hierarchy for duplicate removal for identical duplicates¶
For automated decisions, which apply to identical duplicates, we now define a hierarchy of databases, which decides which record should be kept.
First, list all the original databases:
In [15]:
Copied!
for db in df.originalDatabase.unique():
print(db)
for db in df.originalDatabase.unique():
print(db)
PAGES 2k v2.2.0 FE23 (Breitenmoser et al. (2014)) CoralHydro2k v1.0.1 Iso2k v1.1.2 SISAL v3
Now assign a hierarchy to the original databases. For $n$ original databases the hierarchy ranges from 1, the highest hierarchical value (should always be kept), to the lowest value $n$ (the lowest in the hierarchy)
In [17]:
Copied!
# implement hierarchy for automated decisions for identical records
df = dup.define_hierarchy(df, hierarchy='default')
# implement hierarchy for automated decisions for identical records
df = dup.define_hierarchy(df, hierarchy='default')
Duplicate decision process¶
The following cell takes you through the potential duplicate candidate pairs and lets you decide whether to
- keep both records
- keep just one record
- delete both records
- create composite of both records.
Recollections and updates of duplicates are automatically selected, as well as identical duplicates following the hierarchy defined above.
The output is saved in data/DB/dup_detection/dup_decisions_dod2k_dupfree_INITIALS_DATE.csv
Summary figures are saved in figs/DB/dup_detection/, also linked in the output csv file.
Note: The operator has the option to restart the decision process from a backup file in the directory data/DB/dup_detection. This can be especially useful should the connection be interrupted during the process.
In [18]:
Copied!
df.index
df.index
Out[18]:
RangeIndex(start=0, stop=5320, step=1)
You now have the option to implement an automatic choice for specific database combinations. Please also specify a reason!
This is for records which do not satisfy the hierarchy criterion, i.e. records with different data but identical metadata, such as updated records.
If you do not wish to do this, delete automate_db_choice from kwargs or set to False (default).
In [19]:
Copied!
automate_db_choice = {'preferred_db': 'FE23 (Breitenmoser et al. (2014))',
'rejected_db': 'PAGES 2k v2.2.0',
'reason': 'conservative replication requirement'}
automate_db_choice = {'preferred_db': 'FE23 (Breitenmoser et al. (2014))',
'rejected_db': 'PAGES 2k v2.2.0',
'reason': 'conservative replication requirement'}
In [30]:
Copied!
# remove_identicals = True if you want to automatically remove identical duplicates, without operator input
dup.duplicate_decisions_multiple(df, operator_details=operator_details, choose_recollection=True,
remove_identicals=True, backup=True, comment=True, automate_db_choice=automate_db_choice)
# remove_identicals = True if you want to automatically remove identical duplicates, without operator input
dup.duplicate_decisions_multiple(df, operator_details=operator_details, choose_recollection=True,
remove_identicals=True, backup=True, comment=True, automate_db_choice=automate_db_choice)
header [' Decisions for duplicate candidate pairs. ', ' Operated by Lucie Luecke (LL)', ' E-Mail: ljluec1@st-andrews.ac.uk', ' Created on: 2025-12-11 12:50:32.105195 (UTC)', 'index 1,index 2,figure path,datasetId 1,datasetId 2,originalDatabase 1,originalDatabase 2,geo_siteName 1,geo_siteName 2,geo_meanLat 1,geo_meanLat 2,geo_meanLon 1,geo_meanLon 2,geo_meanElevation 1,geo_meanElevation 2,archiveType 1,archiveType 2,paleoData_proxy 1,paleoData_proxy 2,originalDataURL 1,originalDataURL 2,year 1,year 2,Decision 1,Decision 2,Decision type,Decision comment'] data [['0', '4408', 'no figure', 'pages2k_0', 'iso2k_296', 'PAGES 2k v2.2.0', 'Iso2k v1.1.2', 'WDC05A', 'WDC05A', '-79.45999908447266', '-79.45999908447266', '-112.08999633789062', '-112.08999633789062', '1806.0', '1806.0', 'GlacierIce', 'GlacierIce', 'd18O', 'd18O', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ant-WDC05A.Steig.2013.txt', 'https://www.ncdc.noaa.gov/paleo/study/22531', '786.0-2005.0', '786.0-2005.0', 'REMOVE', 'KEEP', 'AUTO: IDENTICAL except for URLs and/or geo_siteName.', 'RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #2.'], ['0', '4409', 'no figure', 'pages2k_0', 'iso2k_298', 'PAGES 2k v2.2.0', 'Iso2k v1.1.2', 'WDC05A', 'WDC05A', '-79.45999908447266', '-79.45999908447266', '-112.08999633789062', '-112.08999633789062', '1806.0', '1806.0', 'GlacierIce', 'GlacierIce', 'd18O', 'd18O', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ant-WDC05A.Steig.2013.txt', 'https://www.ncdc.noaa.gov/paleo/study/22531', '786.0-2005.0', '786.0-2005.0', 'REMOVE', 'KEEP', 'AUTO: IDENTICAL except for URLs and/or geo_siteName.', 'RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #2.'], ['0', '4410', 'no figure', 'pages2k_0', 'iso2k_299', 'PAGES 2k v2.2.0', 'Iso2k v1.1.2', 'WDC05A', 'WDC05A', '-79.45999908447266', '-79.45999908447266', '-112.08999633789062', '-112.08999633789062', '1806.0', '1806.0', 'GlacierIce', 'GlacierIce', 'd18O', 'd18O', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ant-WDC05A.Steig.2013.txt', 'https://www.ncdc.noaa.gov/paleo/study/22531', '786.0-2005.0', '786.0-2005.0', 'REMOVE', 'KEEP', 'AUTO: IDENTICAL except for URLs and/or geo_siteName.', 'RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #2.'], ['2', '3037', 'no figure', 'pages2k_6', 'FE23_northamerica_usa_az555', 'PAGES 2k v2.2.0', 'FE23 (Breitenmoser et al. (2014))', 'Mt. Lemon', 'Mt.Lemon', '32.5', '32.45000076293945', '-110.80000305175781', '-110.78333282470703', '2700.0', '2700.0', 'Wood', 'Wood', 'ring width', 'ring width', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/NAm-MtLemon.Briffa.2002-1.txt', 'https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/northamerica/usa/az555-noaa.rwl', '1568.0-1983.0', '1633.0-1983.0', 'REMOVE', 'KEEP', 'AUTO: preferred db and metadata identical.', 'Automated choice. Metadata identical, automatically choose FE23 (Breitenmoser et al. (2014)) over PAGES 2k v2.2.0. conservative replication requirement'], ['17', '1587', 'no figure', 'pages2k_50', 'FE23_northamerica_canada_cana091', 'PAGES 2k v2.2.0', 'FE23 (Breitenmoser et al. (2014))', 'Smithers Ski Area', 'SmithersSkiArea', '54.900001525878906', '54.900001525878906', '-127.30000305175781', '-127.25', '1200.0', '1200.0', 'Wood', 'Wood', 'ring width', 'ring width', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/NAm-SmithersSkiArea.Schweingruber.1996-1.txt', 'https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/northamerica/canada/cana091-noaa.rwl', '1680.0-1983.0', '1713.0-1983.0', 'REMOVE', 'KEEP', 'AUTO: preferred db and metadata identical.', 'Automated choice. Metadata identical, automatically choose FE23 (Breitenmoser et al. (2014)) over PAGES 2k v2.2.0. conservative replication requirement'], ['20', '21', 'https://nzero.umd.edu:444/hub/user-redirect/lab/tree/dod2k_v2.0/figs/dup_detection/all_merged005_pages2k_62_pages2k_63__20_21,jpg', 'pages2k_62', 'pages2k_63', 'PAGES 2k v2.2.0', 'PAGES 2k v2.2.0', 'Smithers Ski Area', 'Smithers Ski Area', '54.900001525878906', '54.900001525878906', '-127.30000305175781', '-127.30000305175781', '1200.0', '1200.0', 'Wood', 'Wood', 'maximum latewood density', 'maximum latewood density', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/NAm-SmithersSkiArea.Schweingruber.1996-2.txt', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/NAm-SmithersSkiArea.Schweingruber.1996-2.txt', '1680.0-1983.0', '1680.0-1983.0', 'KEEP', 'KEEP', 'MANUAL', ''], ['29', '4146', 'no figure', 'pages2k_81', 'ch2k_HE08LRA01_76', 'PAGES 2k v2.2.0', 'CoralHydro2k v1.0.1', 'Los Roques, Venezuela', 'Cayo Sal, Los Roques Archipelago, Venezuela', '11.770000457763672', '11.770000457763672', '-66.75', '-66.75', '-2.0', '-2.0', 'Coral', 'Coral', 'd18O', 'd18O', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-LosRoques.Hetzinger.2008.txt', 'https://www.ncdc.noaa.gov/paleo/study/12891', '1917.8-2004.8', '1917.8-2004.8', 'REMOVE', 'KEEP', 'AUTO: IDENTICAL except for URLs and/or geo_siteName.', 'RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #2.'], ['29', '4736', 'no figure', 'pages2k_81', 'iso2k_1813', 'PAGES 2k v2.2.0', 'Iso2k v1.1.2', 'Los Roques, Venezuela', 'Cayo Sal, Los Roques Archipelago, Venezuela', '11.770000457763672', '11.770000457763672', '-66.75', '-66.75', '-2.0', '-2.0', 'Coral', 'Coral', 'd18O', 'd18O', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-LosRoques.Hetzinger.2008.txt', 'https://www.ncdc.noaa.gov/paleo/study/12891', '1917.8-2004.8', '1917.8-2004.8', 'REMOVE', 'KEEP', 'AUTO: IDENTICAL except for URLs and/or geo_siteName.', 'RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #2.'], ['30', '4767', 'no figure', 'pages2k_83', 'iso2k_1916', 'PAGES 2k v2.2.0', 'Iso2k v1.1.2', 'Agassiz', 'Agassiz A79', '80.69999694824219', '80.69999694824219', '-73.0999984741211', '-73.0999984741211', '1700.0', '1700.0', 'GlacierIce', 'GlacierIce', 'd18O', 'd18O', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Arc-Agassiz.Vinther.2008.txt', 'https://www.ncdc.noaa.gov/paleo-search/study/2431', '1.0-1972.0', '1.0-1972.0', 'REMOVE', 'KEEP', 'AUTO: IDENTICAL except for URLs and/or geo_siteName.', 'RECORDS IDENTICAL (identical data) except for metadata. Automatically choose #2.'], ['31', '32', 'no figure', 'pages2k_85', 'pages2k_88', 'PAGES 2k v2.2.0', 'PAGES 2k v2.2.0', 'Laguna Chepical', 'Laguna Chepical', '-32.266700744628906', '-32.266700744628906', '-70.5', '-70.5', '3050.0', '3050.0', 'LakeSediment', 'LakeSediment', 'reflectance', 'reflectance', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/SAm-LagunaChepical.deJong.2013.txt', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/SAm-LagunaChepical.deJong.2013.txt', '1.0-2005.0', '1.0-2005.0', 'KEEP', 'REMOVE', 'AUTO: IDENTICAL', 'RECORDS IDENTICAL (perfect correlation). Automatically choose #1.'], ['34', '1639', 'no figure', 'pages2k_94', 'FE23_northamerica_canada_cana153', 'PAGES 2k v2.2.0', 'FE23 (Breitenmoser et al. (2014))', 'Coppermine River', 'CoppermineRiver', '67.19999694824219', '67.23332977294922', '-115.9000015258789', '-115.91666412353516', '213.0', '213.0', 'Wood', 'Wood', 'ring width', 'ring width', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/NAm-CoppermineRiver.Jacoby.1989.txt', 'https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/northamerica/canada/cana153-noaa.rwl', '1428.0-1977.0', '1508.0-1977.0', 'REMOVE', 'KEEP', 'AUTO: preferred db and metadata identical.', 'Automated choice. Metadata identical, automatically choose FE23 (Breitenmoser et al. (2014)) over PAGES 2k v2.2.0. conservative replication requirement'], ['38', '2809', 'no figure', 'pages2k_107', 'FE23_northamerica_usa_ak046', 'PAGES 2k v2.2.0', 'FE23 (Breitenmoser et al. (2014))', 'Kobuk/Noatak', 'Kobuk/Noatak', '67.0999984741211', '67.06666564941406', '-159.60000610351562', '-159.61666870117188', '100.0', '100.0', 'Wood', 'Wood', 'ring width', 'ring width', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/NAm-KobukNoatak.King.2003.txt', 'https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/northamerica/usa/ak046-noaa.rwl', '978.0-1992.0', '1126.0-1992.0', 'REMOVE', 'KEEP', 'AUTO: preferred db and metadata identical.', 'Automated choice. Metadata identical, automatically choose FE23 (Breitenmoser et al. (2014)) over PAGES 2k v2.2.0. conservative replication requirement'], ['44', '45', 'no figure', 'pages2k_121', 'pages2k_122', 'PAGES 2k v2.2.0', 'PAGES 2k v2.2.0', 'Central Andes composite 9', 'Central Andes composite 9', '-39.33000183105469', '-39.33000183105469', '-71.25', '-71.25', '1100.0', '1100.0', 'Wood', 'Wood', 'ring width', 'ring width', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/SAm-CentralAndes9.Mundo.2014.txt', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/SAm-CentralAndes9.Mundo.2014.txt', '1636.0-2006.0', '1636.0-2006.0', 'KEEP', 'REMOVE', 'AUTO: IDENTICAL', 'RECORDS IDENTICAL (perfect correlation). Automatically choose #1.'], ['50', '1706', 'no figure', 'pages2k_132', 'FE23_northamerica_canada_cana225', 'PAGES 2k v2.2.0', 'FE23 (Breitenmoser et al. (2014))', 'Meadow Mountain', 'MeadowMountain', '50.20000076293945', '50.233333587646484', '-117.0999984741211', '-117.08333587646484', '2100.0', '2100.0', 'Wood', 'Wood', 'ring width', 'ring width', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/NAm-MeadowMountain.Wilson.2005-1.txt', 'https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/northamerica/canada/cana225-noaa.rwl', '1669.0-1997.0', '1707.0-1997.0', 'REMOVE', 'KEEP', 'AUTO: preferred db and metadata identical.', 'Automated choice. Metadata identical, automatically choose FE23 (Breitenmoser et al. (2014)) over PAGES 2k v2.2.0. conservative replication requirement'], ['58', '4018', 'no figure', 'pages2k_158', 'FE23_northamerica_usa_wa069', 'PAGES 2k v2.2.0', 'FE23 (Breitenmoser et al. (2014))', 'Harts Pass N1', "Hart'sPassN1", '48.70000076293945', '48.70000076293945', '-120.69999694824219', '-120.6500015258789', '1925.0', '1925.0', 'Wood', 'Wood', 'ring width', 'ring width', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/NAm-wa069.Peterson.1994.txt', 'https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/northamerica/usa/wa069-noaa.rwl', '1685.0-1991.0', '1724.0-1990.0', 'REMOVE', 'KEEP', 'AUTO: preferred db and metadata identical.', 'Automated choice. Metadata identical, automatically choose FE23 (Breitenmoser et al. (2014)) over PAGES 2k v2.2.0. conservative replication requirement'], ['62', '4102', 'no figure', 'pages2k_171', 'FE23_northamerica_usa_wy021', 'PAGES 2k v2.2.0', 'FE23 (Breitenmoser et al. (2014))', 'Powder River Pass', 'PowderRiverPass', '44.20000076293945', '44.150001525878906', '-107.0999984741211', '-107.05000305175781', '2850.0', '2850.0', 'Wood', 'Wood', 'ring width', 'ring width', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/NAm-PowderRiverPass.Briffa.1996-1.txt', 'https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/northamerica/usa/wy021-noaa.rwl', '1496.0-1983.0', '1684.0-1983.0', 'REMOVE', 'KEEP', 'AUTO: preferred db and metadata identical.', 'Automated choice. Metadata identical, automatically choose FE23 (Breitenmoser et al. (2014)) over PAGES 2k v2.2.0. conservative replication requirement'], ['72', '4526', 'no figure', 'pages2k_203', 'iso2k_826', 'PAGES 2k v2.2.0', 'Iso2k v1.1.2', 'Bunaken Island', 'Bunaken Island', '-1.5', '-1.5', '124.83300018310547', '124.83300018310547', '-3.0', '-3.0', 'Coral', 'Coral', 'd18O', 'd18O', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-BunakenIsland.Charles.2003.txt', 'https://www.ncdc.noaa.gov/paleo/study/1903', '1860.0-1990.6', '1860.0-1990.6', 'REMOVE', 'KEEP', 'AUTO: IDENTICAL except for URLs and/or geo_siteName.', 'RECORDS IDENTICAL (identical data) except for metadata. Automatically choose #2.'], ['82', '3699', 'no figure', 'pages2k_225', 'FE23_northamerica_usa_nv512', 'PAGES 2k v2.2.0', 'FE23 (Breitenmoser et al. (2014))', 'Pearl Peak', 'PearlPeak', '40.20000076293945', '40.233333587646484', '-115.5', '-115.53333282470703', '3170.0', '3170.0', 'Wood', 'Wood', 'ring width', 'ring width', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/NAm-PearlPeak.Graybill.1994.txt', 'https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/northamerica/usa/nv512-noaa.rwl', '320.0-1985.0', '850.0-1985.0', 'REMOVE', 'KEEP', 'AUTO: preferred db and metadata identical.', 'Automated choice. Metadata identical, automatically choose FE23 (Breitenmoser et al. (2014)) over PAGES 2k v2.2.0. conservative replication requirement'], ['86', '4576', 'no figure', 'pages2k_238', 'iso2k_1044', 'PAGES 2k v2.2.0', 'Iso2k v1.1.2', 'Plateau Remote', 'Plateau Remote', '-84.0', '-84.0', '43.0', '43.0', '3330.0', '3330.0', 'GlacierIce', 'GlacierIce', 'd18O', 'd18O', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ant-PlateauRemote.Mosley-Thompson.2013.txt', 'https://www.ncdc.noaa.gov/paleo-search/study/22479', '2.0-1986.0', '2.0-1986.0', 'REMOVE', 'KEEP', 'AUTO: IDENTICAL except for URLs and/or geo_siteName.', 'RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #2.'], ['88', '4328', 'no figure', 'pages2k_242', 'ch2k_LI06FIJ01_582', 'PAGES 2k v2.2.0', 'CoralHydro2k v1.0.1', 'Savusavu Bay', 'Savusavu Bay, Vanua Levu, Fiji', '-16.816699981689453', '-16.81999969482422', '179.23330688476562', '179.22999572753906', '-10.0', '-10.0', 'Coral', 'Coral', 'd18O', 'd18O', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-SavusavuBayAB.Linsley.2006.txt', 'https://www.ncdc.noaa.gov/paleo/study/1003973', '1617.5-2001.5', '1617.5-2001.5', 'REMOVE', 'KEEP', 'AUTO: IDENTICAL except for URLs and/or geo_siteName.', 'RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #2.'], ['88', '4423', 'no figure', 'pages2k_242', 'iso2k_353', 'PAGES 2k v2.2.0', 'Iso2k v1.1.2', 'Savusavu Bay', 'Savusavu Bay, Fiji', '-16.816699981689453', '-16.816699981689453', '179.23330688476562', '179.23330688476562', '-10.0', '-10.0', 'Coral', 'Coral', 'd18O', 'd18O', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-SavusavuBayAB.Linsley.2006.txt', 'https://www.ncdc.noaa.gov/paleo/study/16216', '1617.5-2001.5', '1617.5-2001.5', 'REMOVE', 'KEEP', 'AUTO: IDENTICAL except for URLs and/or geo_siteName.', 'RECORDS IDENTICAL (identical data) except for metadata. Automatically choose #2.'], ['94', '4660', 'no figure', 'pages2k_258', 'iso2k_1498', 'PAGES 2k v2.2.0', 'Iso2k v1.1.2', 'Rarotonga', 'Rarotonga, Cook Islands, South Pacific', '-21.23780059814453', '-21.23780059814453', '-159.8278045654297', '-159.8278045654297', '-18.3', '-18.3', 'Coral', 'Coral', 'd18O', 'd18O', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-Rarotongad18O2R.Linsley.2006.txt', 'https://www.ncdc.noaa.gov/paleo/study/6089', '1726.8-1996.9', '1726.8-1996.9', 'REMOVE', 'KEEP', 'AUTO: IDENTICAL except for URLs and/or geo_siteName.', 'RECORDS IDENTICAL (identical data) except for metadata. Automatically choose #2.'], ['97', '4629', 'no figure', 'pages2k_263', 'iso2k_1322', 'PAGES 2k v2.2.0', 'Iso2k v1.1.2', 'Lombok', 'Lombok Strait, Bali, Indonesia', '-8.247300148010254', '-8.25730037689209', '115.57569885253906', '115.57569885253906', '-3.0', '-3.0', 'Coral', 'Coral', 'd18O', 'd18O', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-Lombok.Charles.2003.txt', 'https://www.ncdc.noaa.gov/paleo/study/1903', '1782.0-1990.0', '1782.0-1990.0', 'REMOVE', 'KEEP', 'AUTO: IDENTICAL except for URLs and/or geo_siteName.', 'RECORDS IDENTICAL (identical data) except for metadata. Automatically choose #2.'], ['99', '4352', 'https://nzero.umd.edu:444/hub/user-redirect/lab/tree/dod2k_v2.0/figs/dup_detection/all_merged023_pages2k_267_iso2k_58__99_4352,jpg', 'pages2k_267', 'iso2k_58', 'PAGES 2k v2.2.0', 'Iso2k v1.1.2', 'Savusavu Bay, Fiji', 'Savusavu Bay, Fiji', '-16.81999969482422', '-16.816699981689453', '179.22999572753906', '179.23330688476562', '-2.0', '-2.0', 'Coral', 'Coral', 'd18O', 'd18O', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-SavusavuBayFiji.Bagnato.2005.txt', 'https://www.ncdc.noaa.gov/paleo/study/1881', '1776.0-2001.0', '1940.0-2000.0', 'REMOVE', 'KEEP', 'MANUAL', ''], ['99', '4581', 'no figure', 'pages2k_267', 'iso2k_1068', 'PAGES 2k v2.2.0', 'Iso2k v1.1.2', 'Savusavu Bay, Fiji', 'Savusavu Bay, Fiji', '-16.81999969482422', '-16.816699981689453', '179.22999572753906', '179.23330688476562', '-2.0', '-2.0', 'Coral', 'Coral', 'd18O', 'd18O', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-SavusavuBayFiji.Bagnato.2005.txt', 'https://www.ncdc.noaa.gov/paleo/study/1916', '1776.0-2001.0', '1776.0-2001.0', 'REMOVE', 'KEEP', 'AUTO: IDENTICAL except for URLs and/or geo_siteName.', 'RECORDS IDENTICAL (identical data) except for metadata. Automatically choose #2.'], ['101', '4306', 'no figure', 'pages2k_271', 'ch2k_FE18RUS01_492', 'PAGES 2k v2.2.0', 'CoralHydro2k v1.0.1', 'Red Sea', 'Ras Umm Sidd, Egypt', '27.850000381469727', '27.84830093383789', '34.31999969482422', '34.310001373291016', '-6.0', '-3.0', 'Coral', 'Coral', 'd18O', 'd18O', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-RedSea.Felis.2000.txt', 'https://doi.pangaea.de/10.1594/PANGAEA.891094', '1751.1-1995.6', '1751.1-1995.6', 'REMOVE', 'KEEP', 'AUTO: IDENTICAL except for URLs and/or geo_siteName.', 'RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #2.'], ['101', '4753', 'no figure', 'pages2k_271', 'iso2k_1861', 'PAGES 2k v2.2.0', 'Iso2k v1.1.2', 'Red Sea', 'Ras Umm Sidd, Red Sea', '27.850000381469727', '27.850000381469727', '34.31999969482422', '34.31999969482422', '-6.0', '-6.0', 'Coral', 'Coral', 'd18O', 'd18O', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-RedSea.Felis.2000.txt', 'https://www.ncdc.noaa.gov/paleo/study/1861', '1751.1-1995.6', '1751.1-1995.6', 'REMOVE', 'KEEP', 'AUTO: IDENTICAL except for URLs and/or geo_siteName.', 'RECORDS IDENTICAL (identical data) except for metadata. Automatically choose #2.'], ['102', '2669', 'no figure', 'pages2k_273', 'FE23_asia_russ130w', 'PAGES 2k v2.2.0', 'FE23 (Breitenmoser et al. (2014))', 'Altai Mt., Jablonsky Pass.', 'JablonskyP.eastAltai', '50.869998931884766', '50.86666488647461', '85.2300033569336', '85.23332977294922', '1450.0', '1450.0', 'Wood', 'Wood', 'ring width', 'ring width', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Asi-AltaiJablonsky.Cook.2000.txt', 'https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/asia/russ130w-noaa.rwl', '1568.0-1995.0', '1669.0-1994.0', 'REMOVE', 'KEEP', 'AUTO: preferred db and metadata identical.', 'Automated choice. Metadata identical, automatically choose FE23 (Breitenmoser et al. (2014)) over PAGES 2k v2.2.0. conservative replication requirement'], ['105', '1641', 'no figure', 'pages2k_281', 'FE23_northamerica_canada_cana155', 'PAGES 2k v2.2.0', 'FE23 (Breitenmoser et al. (2014))', 'Hornby Cabin', 'HornbyCabin', '64.0', '64.03333282470703', '-103.9000015258789', '-103.86666870117188', '160.0', '160.0', 'Wood', 'Wood', 'ring width', 'ring width', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/NAm-HornbyCabin.Jacoby.1989.txt', 'https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/northamerica/canada/cana155-noaa.rwl', '1491.0-1984.0', '1549.0-1984.0', 'REMOVE', 'KEEP', 'AUTO: preferred db and metadata identical.', 'Automated choice. Metadata identical, automatically choose FE23 (Breitenmoser et al. (2014)) over PAGES 2k v2.2.0. conservative replication requirement'], ['109', '2784', 'no figure', 'pages2k_294', 'FE23_northamerica_usa_ak021', 'PAGES 2k v2.2.0', 'FE23 (Breitenmoser et al. (2014))', 'Miners Well', 'MinersWell', '60.0', '60.0', '-141.6999969482422', '-141.68333435058594', '650.0', '650.0', 'Wood', 'Wood', 'ring width', 'ring width', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/NAm-MinersWell.Wiles.2000.txt', 'https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/northamerica/usa/ak021-noaa.rwl', '1428.0-1995.0', '1557.0-1995.0', 'REMOVE', 'KEEP', 'AUTO: preferred db and metadata identical.', 'Automated choice. Metadata identical, automatically choose FE23 (Breitenmoser et al. (2014)) over PAGES 2k v2.2.0. conservative replication requirement'], ['113', '115', 'https://nzero.umd.edu:444/hub/user-redirect/lab/tree/dod2k_v2.0/figs/dup_detection/all_merged030_pages2k_305_pages2k_309__113_115,jpg', 'pages2k_305', 'pages2k_309', 'PAGES 2k v2.2.0', 'PAGES 2k v2.2.0', 'Emerald Basin, Nova Scotia', 'Emerald Basin, Nova Scotia', '43.529998779296875', '43.529998779296875', '-62.47999954223633', '-62.47999954223633', '-250.0', '-250.0', 'MarineSediment', 'MarineSediment', 'alkenone', 'alkenone', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-EmeraldBasin.Keigwin.2007-1.txt', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-EmeraldBasin.Keigwin.2007-2.txt', '400.0-1950.0', '80.0-1950.0', 'KEEP', 'REMOVE', 'MANUAL', ''], ['114', '116', 'https://nzero.umd.edu:444/hub/user-redirect/lab/tree/dod2k_v2.0/figs/dup_detection/all_merged031_pages2k_307_pages2k_311__114_116,jpg', 'pages2k_307', 'pages2k_311', 'PAGES 2k v2.2.0', 'PAGES 2k v2.2.0', 'Emerald Basin, Nova Scotia', 'Emerald Basin, Nova Scotia', '43.529998779296875', '43.529998779296875', '-62.47999954223633', '-62.47999954223633', '-250.0', '-250.0', 'MarineSediment', 'MarineSediment', 'Uk37', 'Uk37', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-EmeraldBasin.Keigwin.2007-1.txt', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-EmeraldBasin.Keigwin.2007-2.txt', '400.0-1950.0', '80.0-1950.0', 'REMOVE', 'KEEP', 'MANUAL', ''], ['118', '4425', 'no figure', 'pages2k_315', 'iso2k_362', 'PAGES 2k v2.2.0', 'Iso2k v1.1.2', 'Austfonna', 'Austfonna', '79.83000183105469', '79.83000183105469', '24.020000457763672', '24.0', '750.0', '750.0', 'GlacierIce', 'GlacierIce', 'd18O', 'd18O', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Arc-Austfonna.Isaksson.2005.txt', 'https://www.ncdc.noaa.gov/paleo-search/study/11173', '1400.0-1998.0', '1400.0-1998.0', 'REMOVE', 'KEEP', 'AUTO: IDENTICAL except for URLs and/or geo_siteName.', 'RECORDS IDENTICAL (identical data) except for metadata. Automatically choose #2.'], ['119', '4148', 'no figure', 'pages2k_317', 'ch2k_NA09MAL01_84', 'PAGES 2k v2.2.0', 'CoralHydro2k v1.0.1', 'Malindi', 'Malindi Marine Park, Kenya', '-3.200000047683716', '-3.200000047683716', '40.099998474121094', '40.099998474121094', '-4.4', '-0.5', 'Coral', 'Coral', 'd18O', 'd18O', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-Malindi.Nakamura.2009.txt', 'https://www.ncdc.noaa.gov/paleo/study/12994', '1887.0-2002.7', '1887.0-2002.7', 'REMOVE', 'KEEP', 'AUTO: IDENTICAL except for URLs and/or geo_siteName.', 'RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #2.'], ['119', '4722', 'no figure', 'pages2k_317', 'iso2k_1754', 'PAGES 2k v2.2.0', 'Iso2k v1.1.2', 'Malindi', 'Malindi', '-3.200000047683716', '-3.200000047683716', '40.099998474121094', '40.099998474121094', '-4.4', '-4.4', 'Coral', 'Coral', 'd18O', 'd18O', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-Malindi.Nakamura.2009.txt', 'https://www.ncdc.noaa.gov/paleo/study/12994', '1887.0-2002.7', '1887.0-2002.7', 'REMOVE', 'KEEP', 'AUTO: IDENTICAL except for URLs and/or geo_siteName.', 'RECORDS IDENTICAL (identical data) except for metadata. Automatically choose #2.'], ['121', '1691', 'no figure', 'pages2k_323', 'FE23_northamerica_canada_cana210', 'PAGES 2k v2.2.0', 'FE23 (Breitenmoser et al. (2014))', 'Medusa Bay', 'MedusaBay', '56.900001525878906', '56.91666793823242', '-61.5', '-61.5', '25.0', '25.0', 'Wood', 'Wood', 'ring width', 'ring width', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/NAm-MedusaBay.Buckley.2003.txt', 'https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/northamerica/canada/cana210-noaa.rwl', '1634.0-1997.0', '1749.0-1997.0', 'REMOVE', 'KEEP', 'AUTO: preferred db and metadata identical.', 'Automated choice. Metadata identical, automatically choose FE23 (Breitenmoser et al. (2014)) over PAGES 2k v2.2.0. conservative replication requirement'], ['142', '4236', 'no figure', 'pages2k_385', 'ch2k_FE09OGA01_304', 'PAGES 2k v2.2.0', 'CoralHydro2k v1.0.1', 'Miyanohama', 'Ogasawara Islands, Japan', '27.105899810791016', '27.105899810791016', '142.19410705566406', '142.19410705566406', '-5.6', '-3.9', 'Coral', 'Coral', 'd18O', 'd18O', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-Miyanohama.Felis.2009.txt', 'https://doi.pangaea.de/10.1594/PANGAEA.743953', '1872.2-1994.9', '1872.2-1994.9', 'REMOVE', 'KEEP', 'AUTO: IDENTICAL except for URLs and/or geo_siteName.', 'RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #2.'], ['142', '4769', 'no figure', 'pages2k_385', 'iso2k_1922', 'PAGES 2k v2.2.0', 'Iso2k v1.1.2', 'Miyanohama', 'Miyanohama', '27.105899810791016', '27.105899810791016', '142.19410705566406', '142.19410705566406', '-5.6', '-5.6', 'Coral', 'Coral', 'd18O', 'd18O', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-Miyanohama.Felis.2009.txt', 'https://www.ncdc.noaa.gov/paleo/study/8608', '1872.2-1994.9', '1872.2-1994.9', 'REMOVE', 'KEEP', 'AUTO: IDENTICAL except for URLs and/or geo_siteName.', 'RECORDS IDENTICAL (identical data) except for metadata. Automatically choose #2.'], ['143', '4237', 'no figure', 'pages2k_387', 'ch2k_FE09OGA01_306', 'PAGES 2k v2.2.0', 'CoralHydro2k v1.0.1', 'Miyanohama', 'Ogasawara Islands, Japan', '27.105899810791016', '27.105899810791016', '142.19410705566406', '142.19410705566406', '-5.6', '-3.9', 'Coral', 'Coral', 'Sr/Ca', 'Sr/Ca', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-Miyanohama.Felis.2009.txt', 'https://doi.pangaea.de/10.1594/PANGAEA.743953', '1872.2-1994.9', '1872.2-1994.9', 'REMOVE', 'KEEP', 'AUTO: IDENTICAL except for URLs and/or geo_siteName.', 'RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #2.'], ['148', '4271', 'no figure', 'pages2k_395', 'ch2k_CA07FLI01_400', 'PAGES 2k v2.2.0', 'CoralHydro2k v1.0.1', 'Coral Sea', 'Flinders Reef, Australia', '-17.729999542236328', '-17.729999542236328', '148.42999267578125', '148.42999267578125', '-5.0', 'nan', 'Coral', 'Coral', 'd18O', 'd18O', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-CoralSea.Calvo.2007.txt', 'https://www.ncdc.noaa.gov/paleo/study/6087', '1708.0-1988.0', '1708.0-1988.0', 'REMOVE', 'KEEP', 'AUTO: IDENTICAL except for URLs and/or geo_siteName.', 'RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #2.'], ['148', '4579', 'no figure', 'pages2k_395', 'iso2k_1057', 'PAGES 2k v2.2.0', 'Iso2k v1.1.2', 'Coral Sea', 'Coral Sea', '-17.729999542236328', '-17.729999542236328', '148.42999267578125', '148.42999267578125', '-5.0', '-5.0', 'Coral', 'Coral', 'd18O', 'd18O', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-CoralSea.Calvo.2007.txt', 'https://www.ncdc.noaa.gov/paleo/study/6087', '1708.0-1988.0', '1708.0-1988.0', 'REMOVE', 'KEEP', 'AUTO: IDENTICAL except for URLs and/or geo_siteName.', 'RECORDS IDENTICAL (identical data) except for metadata. Automatically choose #2.'], ['149', '4272', 'no figure', 'pages2k_397', 'ch2k_CA07FLI01_402', 'PAGES 2k v2.2.0', 'CoralHydro2k v1.0.1', 'Coral Sea', 'Flinders Reef, Australia', '-17.729999542236328', '-17.729999542236328', '148.42999267578125', '148.42999267578125', '-5.0', 'nan', 'Coral', 'Coral', 'Sr/Ca', 'Sr/Ca', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-CoralSea.Calvo.2007.txt', 'https://www.ncdc.noaa.gov/paleo/study/6087', '1708.0-1988.0', '1708.0-1988.0', 'REMOVE', 'KEEP', 'AUTO: IDENTICAL except for URLs and/or geo_siteName.', 'RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #2.'], ['154', '4280', 'no figure', 'pages2k_409', 'ch2k_QU96ESV01_422', 'PAGES 2k v2.2.0', 'CoralHydro2k v1.0.1', 'Vanuatu', 'Espiritu Santo Island, Vanuatu', '-15.0', '-15.0', '166.99000549316406', '167.0', '-1.0', '-1.0', 'Coral', 'Coral', 'd18O', 'd18O', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-Vanuatu.Quinn.1996.txt', 'https://www.ncdc.noaa.gov/paleo/study/1839', '1807.0-1979.0', '1807.0-1979.0', 'REMOVE', 'KEEP', 'AUTO: IDENTICAL except for URLs and/or geo_siteName.', 'RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #2.'], ['154', '4386', 'no figure', 'pages2k_409', 'iso2k_218', 'PAGES 2k v2.2.0', 'Iso2k v1.1.2', 'Vanuatu', 'Tangoa Island, Vanuatu', '-15.0', '-15.0', '166.99000549316406', '167.0', '-1.0', '-1.0', 'Coral', 'Coral', 'd18O', 'd18O', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-Vanuatu.Quinn.1996.txt', 'https://www.ncdc.noaa.gov/paleo/study/1839', '1807.0-1979.0', '1807.0-1979.0', 'REMOVE', 'KEEP', 'AUTO: IDENTICAL except for URLs and/or geo_siteName.', 'RECORDS IDENTICAL (identical data) except for metadata. Automatically choose #2.'], ['156', '158', 'no figure', 'pages2k_414', 'pages2k_418', 'PAGES 2k v2.2.0', 'PAGES 2k v2.2.0', 'West Spitzberg, Fram Strait', 'West Spitzberg, Fram Strait', '78.95999908447266', '78.95999908447266', '5.885000228881836', '5.885000228881836', '-1497.0', '-1497.0', 'MarineSediment', 'MarineSediment', 'temperature', 'temperature', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-WestSpitzberg.Bonnet.2010-2.txt', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-WestSpitzberg.Bonnet.2010-3.txt', '49.0-1942.5', '49.0-1942.5', 'KEEP', 'REMOVE', 'AUTO: IDENTICAL except for URLs and/or geo_siteName.', 'RECORDS IDENTICAL (identical data) except for metadata. Automatically choose #1.'], ['157', '159', 'no figure', 'pages2k_417', 'pages2k_421', 'PAGES 2k v2.2.0', 'PAGES 2k v2.2.0', 'West Spitzberg, Fram Strait', 'West Spitzberg, Fram Strait', '78.95999908447266', '78.95999908447266', '5.885000228881836', '5.885000228881836', '-1497.0', '-1497.0', 'MarineSediment', 'MarineSediment', 'temperature', 'temperature', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-WestSpitzberg.Bonnet.2010-2.txt', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-WestSpitzberg.Bonnet.2010-3.txt', '49.0-1942.5', '49.0-1942.5', 'KEEP', 'REMOVE', 'AUTO: IDENTICAL except for URLs and/or geo_siteName.', 'RECORDS IDENTICAL (identical data) except for metadata. Automatically choose #1.'], ['165', '171', 'no figure', 'pages2k_427', 'pages2k_433', 'PAGES 2k v2.2.0', 'PAGES 2k v2.2.0', 'West Spitzberg, Fram Strait', 'West Spitzberg, Fram Strait', '78.95999908447266', '78.95999908447266', '5.885000228881836', '5.885000228881836', '-1497.0', '-1497.0', 'MarineSediment', 'MarineSediment', 'count', 'count', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-WestSpitzberg.Bonnet.2010-3.txt', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-WestSpitzberg.Bonnet.2010-3.txt', '49.0-1942.5', '49.0-1942.5', 'KEEP', 'REMOVE', 'AUTO: IDENTICAL', 'RECORDS IDENTICAL (perfect correlation). Automatically choose #1.'], ['173', '325', 'no figure', 'pages2k_435', 'pages2k_842', 'PAGES 2k v2.2.0', 'PAGES 2k v2.2.0', 'Altai Mt., Ust Ulagan Lake', 'Altai Mt., Ust Ulagan Lake', '50.47999954223633', '50.47999954223633', '87.6500015258789', '87.6500015258789', '2150.0', '2150.0', 'Wood', 'Wood', 'ring width', 'ring width', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Asi-UULLWD.Schweingruber.2002.txt', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Asi-UULMXD.Schweingruber.2002.txt', '1581.0-1994.0', '1581.0-1994.0', 'KEEP', 'REMOVE', 'AUTO: IDENTICAL except for URLs and/or geo_siteName.', 'RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #1.'], ['176', '177', 'no figure', 'pages2k_444', 'pages2k_445', 'PAGES 2k v2.2.0', 'PAGES 2k v2.2.0', 'Laguna Aculeo', 'Laguna Aculeo', '-33.849998474121094', '-33.849998474121094', '-70.91999816894531', '-70.91999816894531', '355.0', '355.0', 'LakeSediment', 'LakeSediment', 'reflectance', 'reflectance', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/SAm-LagunaAculeo.vonGunten.2009.txt', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/SAm-LagunaAculeo.vonGunten.2009.txt', '856.0-1997.0', '856.0-1997.0', 'KEEP', 'REMOVE', 'AUTO: IDENTICAL', 'RECORDS IDENTICAL (perfect correlation). Automatically choose #1.'], ['176', '178', 'no figure', 'pages2k_444', 'pages2k_446', 'PAGES 2k v2.2.0', 'PAGES 2k v2.2.0', 'Laguna Aculeo', 'Laguna Aculeo', '-33.849998474121094', '-33.849998474121094', '-70.91999816894531', '-70.91999816894531', '355.0', '355.0', 'LakeSediment', 'LakeSediment', 'reflectance', 'reflectance', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/SAm-LagunaAculeo.vonGunten.2009.txt', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/SAm-LagunaAculeo.vonGunten.2009.txt', '856.0-1997.0', '856.0-1997.0', 'KEEP', 'REMOVE', 'AUTO: IDENTICAL', 'RECORDS IDENTICAL (perfect correlation). Automatically choose #1.'], ['177', '178', 'no figure', 'pages2k_445', 'pages2k_446', 'PAGES 2k v2.2.0', 'PAGES 2k v2.2.0', 'Laguna Aculeo', 'Laguna Aculeo', '-33.849998474121094', '-33.849998474121094', '-70.91999816894531', '-70.91999816894531', '355.0', '355.0', 'LakeSediment', 'LakeSediment', 'reflectance', 'reflectance', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/SAm-LagunaAculeo.vonGunten.2009.txt', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/SAm-LagunaAculeo.vonGunten.2009.txt', '856.0-1997.0', '856.0-1997.0', 'KEEP', 'REMOVE', 'AUTO: IDENTICAL', 'RECORDS IDENTICAL (perfect correlation). Automatically choose #1.'], ['184', '4208', 'no figure', 'pages2k_462', 'ch2k_OS14UCP01_236', 'PAGES 2k v2.2.0', 'CoralHydro2k v1.0.1', 'Palau', 'Ulong Channel, Palau', '7.285900115966797', '7.285900115966797', '134.25030517578125', '134.25030517578125', '-12.0', '-12.0', 'Coral', 'Coral', 'd18O', 'd18O', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-PalauUlongChannel.Osborne.2014.txt', 'https://www.ncdc.noaa.gov/paleo/study/16339', '1793.2-2008.2', '1793.2-2008.2', 'REMOVE', 'KEEP', 'AUTO: IDENTICAL except for URLs and/or geo_siteName.', 'RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #2.'], ['184', '4422', 'no figure', 'pages2k_462', 'iso2k_350', 'PAGES 2k v2.2.0', 'Iso2k v1.1.2', 'Palau', 'Palau', '7.285900115966797', '7.285900115966797', '134.25030517578125', '134.25030517578125', '-12.0', '-12.0', 'Coral', 'Coral', 'd18O', 'd18O', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-PalauUlongChannel.Osborne.2014.txt', 'https://www.ncdc.noaa.gov/paleo/study/16339', '1793.2-2008.2', '1793.2-2008.2', 'REMOVE', 'KEEP', 'AUTO: IDENTICAL except for URLs and/or geo_siteName.', 'RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #2.'], ['187', '1310', 'no figure', 'pages2k_468', 'pages2k_3550', 'PAGES 2k v2.2.0', 'PAGES 2k v2.2.0', 'Altai Mt., Ust Ulagan Lake', 'Altai Mt., Ust Ulagan Lake', '50.47999954223633', '50.47999954223633', '87.6500015258789', '87.6500015258789', '2150.0', '2150.0', 'Wood', 'Wood', 'ring width', 'ring width', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Asi-UULTRW.Schweingruber.2002.txt', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Asi-UULEWW.Schweingruber.2002.txt', '1581.0-1994.0', '1581.0-1994.0', 'KEEP', 'REMOVE', 'AUTO: IDENTICAL except for URLs and/or geo_siteName.', 'RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #1.'], ['187', '2676', 'no figure', 'pages2k_468', 'FE23_asia_russ137w', 'PAGES 2k v2.2.0', 'FE23 (Breitenmoser et al. (2014))', 'Altai Mt., Ust Ulagan Lake', 'UstUlaganLake(Altai)', '50.47999954223633', '50.483333587646484', '87.6500015258789', '87.6500015258789', '2150.0', '2150.0', 'Wood', 'Wood', 'ring width', 'ring width', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Asi-UULTRW.Schweingruber.2002.txt', 'https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/asia/russ137w-noaa.rwl', '1581.0-1994.0', '1783.0-1994.0', 'REMOVE', 'KEEP', 'AUTO: preferred db and metadata identical.', 'Automated choice. Metadata identical, automatically choose FE23 (Breitenmoser et al. (2014)) over PAGES 2k v2.2.0. conservative replication requirement'], ['189', '190', 'no figure', 'pages2k_472', 'pages2k_474', 'PAGES 2k v2.2.0', 'PAGES 2k v2.2.0', 'Fisk Basin, Gulf of Mexico', 'Fisk Basin, Gulf of Mexico', '27.549999237060547', '27.549999237060547', '-93.93000030517578', '-93.93000030517578', '-817.0', '-817.0', 'MarineSediment', 'MarineSediment', 'Mg/Ca', 'Mg/Ca', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-FiskBasin.Richey.2009-1.txt', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-FiskBasin.Richey.2009-1.txt', '1216.8-1950.0', '1216.8-1950.0', 'KEEP', 'REMOVE', 'AUTO: IDENTICAL', 'RECORDS IDENTICAL (perfect correlation). Automatically choose #1.'], ['189', '192', 'https://nzero.umd.edu:444/hub/user-redirect/lab/tree/dod2k_v2.0/figs/dup_detection/all_merged056_pages2k_472_pages2k_477__189_192,jpg', 'pages2k_472', 'pages2k_477', 'PAGES 2k v2.2.0', 'PAGES 2k v2.2.0', 'Fisk Basin, Gulf of Mexico', 'Fisk Basin, Gulf of Mexico', '27.549999237060547', '27.549999237060547', '-93.93000030517578', '-93.93000030517578', '-817.0', '-817.0', 'MarineSediment', 'MarineSediment', 'Mg/Ca', 'Mg/Ca', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-FiskBasin.Richey.2009-1.txt', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-FiskBasin.Richey.2009-2.txt', '1216.8-1950.0', '1198.0-1950.0', 'KEEP', 'KEEP', 'MANUAL', ''], ['190', '192', 'https://nzero.umd.edu:444/hub/user-redirect/lab/tree/dod2k_v2.0/figs/dup_detection/all_merged057_pages2k_474_pages2k_477__190_192,jpg', 'pages2k_474', 'pages2k_477', 'PAGES 2k v2.2.0', 'PAGES 2k v2.2.0', 'Fisk Basin, Gulf of Mexico', 'Fisk Basin, Gulf of Mexico', '27.549999237060547', '27.549999237060547', '-93.93000030517578', '-93.93000030517578', '-817.0', '-817.0', 'MarineSediment', 'MarineSediment', 'Mg/Ca', 'Mg/Ca', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-FiskBasin.Richey.2009-1.txt', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-FiskBasin.Richey.2009-2.txt', '1216.8-1950.0', '1198.0-1950.0', 'KEEP', 'REMOVE', 'MANUAL', ''], ['193', '4744', 'no figure', 'pages2k_478', 'iso2k_1846', 'PAGES 2k v2.2.0', 'Iso2k v1.1.2', 'Dye', 'DYE3', '65.18000030517578', '65.18000030517578', '-43.83000183105469', '-43.83000183105469', '2480.0', '2480.0', 'GlacierIce', 'GlacierIce', 'd18O', 'd18O', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Arc-Dye.Vinther.2010.txt', 'http://www.iceandclimate.nbi.ku.dk/data/Vinther_etal_2010_data_02feb2010.xls', '1.0-1978.0', '1.0-1978.0', 'REMOVE', 'KEEP', 'AUTO: IDENTICAL except for URLs and/or geo_siteName.', 'RECORDS IDENTICAL (identical data) except for metadata. Automatically choose #2.'], ['196', '3157', 'no figure', 'pages2k_486', 'FE23_northamerica_usa_ca609', 'PAGES 2k v2.2.0', 'FE23 (Breitenmoser et al. (2014))', 'Fish Creek Trail (San Gorgonio)', 'FishCreekTrail(SanGorgonio)', '34.099998474121094', '34.11666488647461', '-116.80000305175781', '-116.80000305175781', '2890.0', '2890.0', 'Wood', 'Wood', 'ring width', 'ring width', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/NAm-FishCreekTrail.Biondi.2001.txt', 'https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/northamerica/usa/ca609-noaa.rwl', '1534.0-1995.0', '1708.0-1995.0', 'REMOVE', 'KEEP', 'AUTO: preferred db and metadata identical.', 'Automated choice. Metadata identical, automatically choose FE23 (Breitenmoser et al. (2014)) over PAGES 2k v2.2.0. conservative replication requirement'], ['199', '4123', 'no figure', 'pages2k_495', 'ch2k_LI06RAR01_12', 'PAGES 2k v2.2.0', 'CoralHydro2k v1.0.1', 'Rarotonga', 'Rarotonga, Cook Islands', '-21.23780059814453', '-21.23780059814453', '-159.8278045654297', '-159.8278045654297', '-18.0', '-18.0', 'Coral', 'Coral', 'd18O', 'd18O', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-Rarotongad18O99.Linsley.2006.txt', 'https://www.ncdc.noaa.gov/paleo/study/6089', '1906.9-1999.8', '1906.9-1999.8', 'REMOVE', 'KEEP', 'AUTO: IDENTICAL except for URLs and/or geo_siteName.', 'RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #2.'], ['199', '4662', 'no figure', 'pages2k_495', 'iso2k_1502', 'PAGES 2k v2.2.0', 'Iso2k v1.1.2', 'Rarotonga', 'Rarotonga, Cook Islands, South Pacific', '-21.23780059814453', '-21.23780059814453', '-159.8278045654297', '-159.8278045654297', '-18.0', '-18.3', 'Coral', 'Coral', 'd18O', 'd18O', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-Rarotongad18O99.Linsley.2006.txt', 'https://www.ncdc.noaa.gov/paleo/study/6089', '1906.9-1999.8', '1906.9-1999.8', 'REMOVE', 'KEEP', 'AUTO: IDENTICAL except for URLs and/or geo_siteName.', 'RECORDS IDENTICAL (identical data) except for metadata. Automatically choose #2.'], ['202', '4235', 'no figure', 'pages2k_500', 'ch2k_AS05GUA01_302', 'PAGES 2k v2.2.0', 'CoralHydro2k v1.0.1', 'Double Reef', 'Double Reef, Guam', '13.598199844360352', '13.597999572753906', '144.83590698242188', '144.83599853515625', '-7.8', '-7.8', 'Coral', 'Coral', 'd18O', 'd18O', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-DoubleReef.Asami.2005.txt', 'https://www.ncdc.noaa.gov/paleo/study/1915', '1790.0-2000.2', '1790.0-2000.2', 'REMOVE', 'KEEP', 'AUTO: IDENTICAL except for URLs and/or geo_siteName.', 'RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #2.'], ['202', '4675', 'no figure', 'pages2k_500', 'iso2k_1559', 'PAGES 2k v2.2.0', 'Iso2k v1.1.2', 'Double Reef', 'Guam Coral', '13.598199844360352', '13.597999572753906', '144.83590698242188', '144.83599853515625', '-7.8', '-7.8', 'Coral', 'Coral', 'd18O', 'd18O', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-DoubleReef.Asami.2005.txt', 'https://www.ncdc.noaa.gov/paleo/study/1915', '1790.0-2000.2', '1790.0-2000.2', 'REMOVE', 'KEEP', 'AUTO: IDENTICAL except for URLs and/or geo_siteName.', 'RECORDS IDENTICAL (identical data) except for metadata. Automatically choose #2.'], ['216', '4431', 'https://nzero.umd.edu:444/hub/user-redirect/lab/tree/dod2k_v2.0/figs/dup_detection/all_merged064_pages2k_541_iso2k_404__216_4431,jpg', 'pages2k_541', 'iso2k_404', 'PAGES 2k v2.2.0', 'Iso2k v1.1.2', 'MES', 'Mt Erebus Saddle - MES', '-77.51499938964844', '-77.5199966430664', '167.67649841308594', '167.67999267578125', '1600.0', '1600.0', 'GlacierIce', 'GlacierIce', 'dD', 'dD', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ant-MES.Rhodes.2012.txt', 'https://www.ncdc.noaa.gov/paleo/study/13175', '1472.6-2006.8', '1472.6-2006.9', 'KEEP', 'REMOVE', 'MANUAL', ''], ['217', '373', 'no figure', 'pages2k_543', 'pages2k_976', 'PAGES 2k v2.2.0', 'PAGES 2k v2.2.0', 'Altai Mt., Ust Koksa Hill', 'Altai Mt., Ust Koksa Hill', '50.150001525878906', '50.150001525878906', '85.37000274658203', '85.37000274658203', '1750.0', '1750.0', 'Wood', 'Wood', 'ring width', 'ring width', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Asi-UKHMXD.Schweingruber.2002.txt', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Asi-UKHLWD.Schweingruber.2002.txt', '1581.0-1994.0', '1581.0-1994.0', 'KEEP', 'REMOVE', 'AUTO: IDENTICAL except for URLs and/or geo_siteName.', 'RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #1.'], ['224', '4568', 'no figure', 'pages2k_565', 'iso2k_998', 'PAGES 2k v2.2.0', 'Iso2k v1.1.2', 'Penny Ice Cap P96', 'Penny Ice cap', '67.25', '67.25', '-66.75', '-66.75', '1810.0', '1900.0', 'GlacierIce', 'GlacierIce', 'd18O', 'd18O', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Arc-PennyIceCapP96.Fisher.1998.txt', 'www.ncdc.noaa.gov/paleo/study/2474', '5.0-1980.0', '5.0-1980.0', 'REMOVE', 'KEEP', 'AUTO: IDENTICAL except for URLs and/or geo_siteName.', 'RECORDS IDENTICAL (identical data) except for metadata. Automatically choose #2.'], ['233', '3550', 'no figure', 'pages2k_583', 'FE23_northamerica_usa_mt116', 'PAGES 2k v2.2.0', 'FE23 (Breitenmoser et al. (2014))', 'Flint Creek Range', 'FlintCreekRange', '46.29999923706055', '46.28333282470703', '-113.19999694824219', '-113.1500015258789', '2645.0', '2645.0', 'Wood', 'Wood', 'ring width', 'ring width', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/NAm-FlintCreekRange.Hughes.2005.txt', 'https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/northamerica/usa/mt116-noaa.rwl', '999.0-1998.0', '1083.0-1998.0', 'REMOVE', 'KEEP', 'AUTO: preferred db and metadata identical.', 'Automated choice. Metadata identical, automatically choose FE23 (Breitenmoser et al. (2014)) over PAGES 2k v2.2.0. conservative replication requirement'], ['236', '4221', 'no figure', 'pages2k_592', 'ch2k_LI06RAR02_270', 'PAGES 2k v2.2.0', 'CoralHydro2k v1.0.1', 'Rarotonga', 'Rarotonga, Cook Islands', '-21.23780059814453', '-21.23780059814453', '-159.8278045654297', '-159.8278045654297', '-10.0', '-10.0', 'Coral', 'Coral', 'd18O', 'd18O', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-Rarotongad18O3R.Linsley.2006.txt', 'https://www.ncdc.noaa.gov/paleo/study/6089', '1874.7-2000.2', '1874.7-2000.2', 'REMOVE', 'KEEP', 'AUTO: IDENTICAL except for URLs and/or geo_siteName.', 'RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #2.'], ['236', '4661', 'no figure', 'pages2k_592', 'iso2k_1500', 'PAGES 2k v2.2.0', 'Iso2k v1.1.2', 'Rarotonga', 'Rarotonga, Cook Islands, South Pacific', '-21.23780059814453', '-21.23780059814453', '-159.8278045654297', '-159.8278045654297', '-10.0', '-18.3', 'Coral', 'Coral', 'd18O', 'd18O', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-Rarotongad18O3R.Linsley.2006.txt', 'https://www.ncdc.noaa.gov/paleo/study/6089', '1874.7-2000.2', '1874.7-2000.2', 'REMOVE', 'KEEP', 'AUTO: IDENTICAL except for URLs and/or geo_siteName.', 'RECORDS IDENTICAL (identical data) except for metadata. Automatically choose #2.'], ['243', '4601', 'no figure', 'pages2k_610', 'iso2k_1199', 'PAGES 2k v2.2.0', 'Iso2k v1.1.2', 'Ferrigno', 'Ferrigno', '-74.56999969482422', '-74.56999969482422', '-86.9000015258789', '-86.9000015258789', '1354.0', '1354.0', 'GlacierIce', 'GlacierIce', 'dD', 'dD', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ant-Ferrigno.Thomas.2013.txt', 'https://www.ncdc.noaa.gov/paleo/study/22477', '1703.0-2010.0', '1703.0-2010.0', 'REMOVE', 'KEEP', 'AUTO: IDENTICAL except for URLs and/or geo_siteName.', 'RECORDS IDENTICAL (identical data) except for metadata. Automatically choose #2.'], ['250', '4020', 'no figure', 'pages2k_626', 'FE23_northamerica_usa_wa071', 'PAGES 2k v2.2.0', 'FE23 (Breitenmoser et al. (2014))', 'Harts Pass N1', "Hart'sPassN1", '48.70000076293945', '48.70000076293945', '-120.69999694824219', '-120.6500015258789', '1925.0', '1925.0', 'Wood', 'Wood', 'ring width', 'ring width', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/NAm-wa071.Peterson.1994.txt', 'https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/northamerica/usa/wa071-noaa.rwl', '1585.0-1991.0', '1668.0-1990.0', 'REMOVE', 'KEEP', 'AUTO: preferred db and metadata identical.', 'Automated choice. Metadata identical, automatically choose FE23 (Breitenmoser et al. (2014)) over PAGES 2k v2.2.0. conservative replication requirement'], ['272', '1558', 'no figure', 'pages2k_691', 'FE23_northamerica_canada_cana062', 'PAGES 2k v2.2.0', 'FE23 (Breitenmoser et al. (2014))', 'Lac Romanel (Feucht)', 'LacRomanel(Feucht)', '56.20000076293945', '56.233333587646484', '-67.69999694824219', '-67.71666717529297', '1000.0', '1000.0', 'Wood', 'Wood', 'ring width', 'ring width', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/NAm-LacRomanel.Schweingruber.1996-1.txt', 'https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/northamerica/canada/cana062-noaa.rwl', '1659.0-1988.0', '1755.0-1988.0', 'REMOVE', 'KEEP', 'AUTO: preferred db and metadata identical.', 'Automated choice. Metadata identical, automatically choose FE23 (Breitenmoser et al. (2014)) over PAGES 2k v2.2.0. conservative replication requirement'], ['285', '4429', 'no figure', 'pages2k_730', 'iso2k_396', 'PAGES 2k v2.2.0', 'Iso2k v1.1.2', 'Spannagel Cave,', 'Spannagel Cave, Austria', '47.099998474121094', '47.09000015258789', '11.600000381469727', '11.670000076293945', '2347.0', '2531.0', 'Speleothem', 'Speleothem', 'd18O', 'd18O', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Eur-SpannagelCave.Mangini.2005.txt', 'https://www.ncdc.noaa.gov/paleo/study/5433', '5.0-1935.0', '5.0-1935.0', 'REMOVE', 'KEEP', 'AUTO: IDENTICAL except for URLs and/or geo_siteName.', 'RECORDS IDENTICAL (identical data) except for metadata. Automatically choose #2.'], ['287', '4105', 'no figure', 'pages2k_736', 'FE23_northamerica_usa_wy024', 'PAGES 2k v2.2.0', 'FE23 (Breitenmoser et al. (2014))', 'Granite Pass Hunt Mountain', 'GranitePassHuntMountain', '44.79999923706055', '44.78333282470703', '-107.9000015258789', '-107.86666870117188', '2820.0', '2820.0', 'Wood', 'Wood', 'ring width', 'ring width', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/NAm-GranitePassHuntMountain.Briffa.1996-1.txt', 'https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/northamerica/usa/wy024-noaa.rwl', '1508.0-1983.0', '1708.0-1983.0', 'REMOVE', 'KEEP', 'AUTO: preferred db and metadata identical.', 'Automated choice. Metadata identical, automatically choose FE23 (Breitenmoser et al. (2014)) over PAGES 2k v2.2.0. conservative replication requirement'], ['307', '1715', 'no figure', 'pages2k_800', 'FE23_northamerica_canada_cana234', 'PAGES 2k v2.2.0', 'FE23 (Breitenmoser et al. (2014))', 'Big White 2', 'BigWhite2', '49.70000076293945', '49.733333587646484', '-118.9000015258789', '-118.93333435058594', '2000.0', '2000.0', 'Wood', 'Wood', 'ring width', 'ring width', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/NAm-BigWhite2.Wilson.2005-2.txt', 'https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/northamerica/canada/cana234-noaa.rwl', '1580.0-1997.0', '1666.0-1997.0', 'REMOVE', 'KEEP', 'AUTO: preferred db and metadata identical.', 'Automated choice. Metadata identical, automatically choose FE23 (Breitenmoser et al. (2014)) over PAGES 2k v2.2.0. conservative replication requirement'], ['312', '4451', 'no figure', 'pages2k_818', 'iso2k_488', 'PAGES 2k v2.2.0', 'Iso2k v1.1.2', 'Puruogangri', 'Puruogangri ice cap', '33.91669845581055', '33.91669845581055', '89.08329772949219', '89.08329772949219', '6070.0', '6070.0', 'GlacierIce', 'GlacierIce', 'd18O', 'd18O', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Asi-Puruogangri.Thompson.2006.txt', 'nan', '4.5-1994.5', '4.5-1994.5', 'REMOVE', 'KEEP', 'AUTO: IDENTICAL except for URLs and/or geo_siteName.', 'RECORDS IDENTICAL (identical data) except for metadata. Automatically choose #2.'], ['317', '319', 'no figure', 'pages2k_827', 'pages2k_830', 'PAGES 2k v2.2.0', 'PAGES 2k v2.2.0', 'CH07-98-MC-22, Carolina Slope, western North Atlantic', 'CH07-98-MC-22, Carolina Slope, western North Atlantic', '32.784000396728516', '32.784000396728516', '-76.2760009765625', '-76.2760009765625', '-1895.0', '-1895.0', 'MarineSediment', 'MarineSediment', 'Mg/Ca', 'Mg/Ca', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-CH07-98-MC-22.Saenger.2011-1.txt', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-CH07-98-MC-22.Saenger.2011-2.txt', '250.0-1850.0', '250.0-1850.0', 'KEEP', 'REMOVE', 'AUTO: IDENTICAL except for URLs and/or geo_siteName.', 'RECORDS IDENTICAL (identical data) except for metadata. Automatically choose #1.'], ['320', '813', 'no figure', 'pages2k_831', 'pages2k_2220', 'PAGES 2k v2.2.0', 'PAGES 2k v2.2.0', 'Altai Mt., Ust Koksa Hill', 'Altai Mt., Ust Koksa Hill', '50.150001525878906', '50.150001525878906', '85.37000274658203', '85.37000274658203', '1750.0', '1750.0', 'Wood', 'Wood', 'ring width', 'ring width', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Asi-UKHTRW.Schweingruber.2002.txt', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Asi-UKHEWW.Schweingruber.2002.txt', '1581.0-1994.0', '1581.0-1994.0', 'KEEP', 'REMOVE', 'AUTO: IDENTICAL except for URLs and/or geo_siteName.', 'RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #1.'], ['320', '2666', 'no figure', 'pages2k_831', 'FE23_asia_russ127w', 'PAGES 2k v2.2.0', 'FE23 (Breitenmoser et al. (2014))', 'Altai Mt., Ust Koksa Hill', 'UstKoksaHill(Altai)', '50.150001525878906', '50.150001525878906', '85.37000274658203', '85.36666870117188', '1750.0', '1750.0', 'Wood', 'Wood', 'ring width', 'ring width', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Asi-UKHTRW.Schweingruber.2002.txt', 'https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/asia/russ127w-noaa.rwl', '1581.0-1994.0', '1613.0-1994.0', 'REMOVE', 'KEEP', 'AUTO: preferred db and metadata identical.', 'Automated choice. Metadata identical, automatically choose FE23 (Breitenmoser et al. (2014)) over PAGES 2k v2.2.0. conservative replication requirement'], ['331', '3909', 'no figure', 'pages2k_857', 'FE23_northamerica_usa_ut511', 'PAGES 2k v2.2.0', 'FE23 (Breitenmoser et al. (2014))', 'Ceader Breaks', 'CeaderBreaks', '37.599998474121094', '37.58333206176758', '-113.9000015258789', '-113.8499984741211', '3120.0', '3120.0', 'Wood', 'Wood', 'ring width', 'ring width', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/NAm-CeaderBreaks.Briffa.1996-1.txt', 'https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/northamerica/usa/ut511-noaa.rwl', '1581.0-1983.0', '1653.0-1983.0', 'REMOVE', 'KEEP', 'AUTO: preferred db and metadata identical.', 'Automated choice. Metadata identical, automatically choose FE23 (Breitenmoser et al. (2014)) over PAGES 2k v2.2.0. conservative replication requirement'], ['339', '4570', 'no figure', 'pages2k_881', 'iso2k_1010', 'PAGES 2k v2.2.0', 'Iso2k v1.1.2', 'Malindi', 'Malindi Marine Park', '-3.200000047683716', '-3.2555999755859375', '40.099998474121094', '40.1432991027832', '-6.0', '-6.0', 'Coral', 'Coral', 'd18O', 'd18O', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-Malindi.Cole.2000.txt', 'https://www.ncdc.noaa.gov/paleo/study/1855', '1801.0-1994.0', '1801.0-1994.0', 'REMOVE', 'KEEP', 'AUTO: IDENTICAL except for URLs and/or geo_siteName.', 'RECORDS IDENTICAL (identical data) except for metadata. Automatically choose #2.'], ['342', '343', 'no figure', 'pages2k_893', 'pages2k_895', 'PAGES 2k v2.2.0', 'PAGES 2k v2.2.0', 'Feni Drift', 'Feni Drift', '55.5', '55.5', '-13.899999618530273', '-13.899999618530273', '-2543.0', '-2543.0', 'MarineSediment', 'MarineSediment', 'Mg/Ca', 'Mg/Ca', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-FeniDrift.Richter.2009-1.txt', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-FeniDrift.Richter.2009-1.txt', '1.0-1998.0', '1.0-1998.0', 'KEEP', 'REMOVE', 'AUTO: IDENTICAL', 'RECORDS IDENTICAL (perfect correlation). Automatically choose #1.'], ['342', '345', 'no figure', 'pages2k_893', 'pages2k_900', 'PAGES 2k v2.2.0', 'PAGES 2k v2.2.0', 'Feni Drift', 'Feni Drift', '55.5', '55.5', '-13.899999618530273', '-13.899999618530273', '-2543.0', '-2543.0', 'MarineSediment', 'MarineSediment', 'Mg/Ca', 'Mg/Ca', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-FeniDrift.Richter.2009-1.txt', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-FeniDrift.Richter.2009-2.txt', '1.0-1998.0', '1.0-1998.0', 'KEEP', 'REMOVE', 'AUTO: IDENTICAL except for URLs and/or geo_siteName.', 'RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #1.'], ['343', '345', 'no figure', 'pages2k_895', 'pages2k_900', 'PAGES 2k v2.2.0', 'PAGES 2k v2.2.0', 'Feni Drift', 'Feni Drift', '55.5', '55.5', '-13.899999618530273', '-13.899999618530273', '-2543.0', '-2543.0', 'MarineSediment', 'MarineSediment', 'Mg/Ca', 'Mg/Ca', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-FeniDrift.Richter.2009-1.txt', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-FeniDrift.Richter.2009-2.txt', '1.0-1998.0', '1.0-1998.0', 'KEEP', 'REMOVE', 'AUTO: IDENTICAL except for URLs and/or geo_siteName.', 'RECORDS IDENTICAL (identical data) except for metadata. Automatically choose #1.'], ['358', '4219', 'no figure', 'pages2k_940', 'ch2k_DR99ABR01_264', 'PAGES 2k v2.2.0', 'CoralHydro2k v1.0.1', 'Abraham Reef', 'Abraham Reef, Australia', '-22.100000381469727', '-22.100000381469727', '153.0', '153.0', '-2.5', '-10.0', 'Coral', 'Coral', 'd18O', 'd18O', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-AbrahamReef.Druffel.1999.txt', 'https://www.ncdc.noaa.gov/paleo/study/1911', '1638.3-1983.3', '1638.3-1983.3', 'REMOVE', 'KEEP', 'AUTO: IDENTICAL except for URLs and/or geo_siteName.', 'RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #2.'], ['358', '4220', 'no figure', 'pages2k_940', 'ch2k_DR99ABR01_266', 'PAGES 2k v2.2.0', 'CoralHydro2k v1.0.1', 'Abraham Reef', 'Abraham Reef, Australia', '-22.100000381469727', '-22.100000381469727', '153.0', '153.0', '-2.5', '-10.0', 'Coral', 'Coral', 'd18O', 'd18O', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-AbrahamReef.Druffel.1999.txt', 'https://www.ncdc.noaa.gov/paleo/study/1911', '1638.3-1983.3', '1638.3-1983.3', 'REMOVE', 'KEEP', 'AUTO: IDENTICAL except for URLs and/or geo_siteName.', 'RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #2.'], ['358', '4361', 'no figure', 'pages2k_940', 'iso2k_91', 'PAGES 2k v2.2.0', 'Iso2k v1.1.2', 'Abraham Reef', 'Abraham Reef, Great Barrier Reef, Australia', '-22.100000381469727', '-22.100000381469727', '153.0', '153.0', '-2.5', '-10.0', 'Coral', 'Coral', 'd18O', 'd18O', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-AbrahamReef.Druffel.1999.txt', 'https://www.ncdc.noaa.gov/paleo/study/1911', '1638.3-1983.3', '1638.3-1983.3', 'REMOVE', 'KEEP', 'AUTO: IDENTICAL except for URLs and/or geo_siteName.', 'RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #2.'], ['361', '4364', 'no figure', 'pages2k_945', 'iso2k_100', 'PAGES 2k v2.2.0', 'Iso2k v1.1.2', 'Coastal DML', 'IND 22B4 Coastal DML', '-70.86000061035156', '-70.86000061035156', '11.539999961853027', '11.539999961853027', '500.0', '703.0', 'GlacierIce', 'GlacierIce', 'd18O', 'd18O', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ant-CoastalDML.Thamban.2012.txt', 'https://www.ncdc.noaa.gov/paleo-search/study/22589', '1533.0-1994.0', '1533.0-1994.0', 'REMOVE', 'KEEP', 'AUTO: IDENTICAL except for URLs and/or geo_siteName.', 'RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #2.'], ['366', '4494', 'no figure', 'pages2k_960', 'iso2k_641', 'PAGES 2k v2.2.0', 'Iso2k v1.1.2', 'Florida Bay', 'Lignumvitae Basin, Florida Bay', '24.93000030517578', '24.91670036315918', '-80.75', '-80.75', '-3.0', '-5.0', 'Coral', 'Coral', 'd18O', 'd18O', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-FloridaBay.Swart.1996.txt', 'https://www.ncdc.noaa.gov/paleo/study/1886', '1824.0-1985.0', '1824.0-1985.0', 'REMOVE', 'KEEP', 'AUTO: IDENTICAL except for URLs and/or geo_siteName.', 'RECORDS IDENTICAL (identical data) except for metadata. Automatically choose #2.'], ['375', '3767', 'no figure', 'pages2k_982', 'FE23_northamerica_usa_or042', 'PAGES 2k v2.2.0', 'FE23 (Breitenmoser et al. (2014))', 'Crater Lake NE', 'CraterLakeNE-Medford', '43.0', '42.96666717529297', '-122.19999694824219', '-122.16666412353516', '2200.0', '2200.0', 'Wood', 'Wood', 'ring width', 'ring width', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/NAm-CraterLakeNE.Briffa.2002-1.txt', 'https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/northamerica/usa/or042-noaa.rwl', '1564.0-1983.0', '1668.0-1983.0', 'REMOVE', 'KEEP', 'AUTO: preferred db and metadata identical.', 'Automated choice. Metadata identical, automatically choose FE23 (Breitenmoser et al. (2014)) over PAGES 2k v2.2.0. conservative replication requirement'], ['382', '4495', 'no figure', 'pages2k_1004', 'iso2k_644', 'PAGES 2k v2.2.0', 'Iso2k v1.1.2', 'Dome F 1993', 'Dome F 1993', '-77.31999969482422', '-77.31999969482422', '39.70000076293945', '39.70000076293945', '3810.0', '3810.0', 'GlacierIce', 'GlacierIce', 'd18O', 'd18O', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ant-DomeF1993.Uemura.2014.txt', 'https://www.ncdc.noaa.gov/paleo-search/study/22471', '423.8-1467.2', '423.8-1467.2', 'REMOVE', 'KEEP', 'AUTO: IDENTICAL except for URLs and/or geo_siteName.', 'RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #2.'], ['389', '3035', 'no figure', 'pages2k_1026', 'FE23_northamerica_usa_az553', 'PAGES 2k v2.2.0', 'FE23 (Breitenmoser et al. (2014))', 'Snow Bowl San Francisco Peak', 'SnowBowlSanFranciscoPeak', '35.400001525878906', '35.43333435058594', '-110.19999694824219', '-110.19999694824219', '3150.0', '3150.0', 'Wood', 'Wood', 'ring width', 'ring width', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/NAm-SnowBowlSanFranciscoPeak.Briffa.2002-1.txt', 'https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/northamerica/usa/az553-noaa.rwl', '1453.0-1983.0', '1640.0-1983.0', 'REMOVE', 'KEEP', 'AUTO: preferred db and metadata identical.', 'Automated choice. Metadata identical, automatically choose FE23 (Breitenmoser et al. (2014)) over PAGES 2k v2.2.0. conservative replication requirement'], ['396', '4605', 'no figure', 'pages2k_1048', 'iso2k_1212', 'PAGES 2k v2.2.0', 'Iso2k v1.1.2', 'VLG', 'VLG', '-77.3302001953125', '-77.33000183105469', '162.533203125', '162.52999877929688', '625.0', '626.0', 'GlacierIce', 'GlacierIce', 'dD', 'dD', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ant-VLG.Bertler.2011.txt', 'https://doi.pangaea.de/10.1594/PANGAEA.866368', '1140.0-2000.0', '1140.0-2000.0', 'REMOVE', 'KEEP', 'AUTO: IDENTICAL except for URLs and/or geo_siteName.', 'RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #2.'], ['407', '3547', 'no figure', 'pages2k_1089', 'FE23_northamerica_usa_mt112', 'PAGES 2k v2.2.0', 'FE23 (Breitenmoser et al. (2014))', 'Yellow Mountain Ridge', 'YellowMountainRidge1', '45.29999923706055', '45.29999923706055', '-111.30000305175781', '-111.31666564941406', '2500.0', '2500.0', 'Wood', 'Wood', 'ring width', 'ring width', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/NAm-YellowMountainRidge.King.2002.txt', 'https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/northamerica/usa/mt112-noaa.rwl', '470.0-1998.0', '850.0-1998.0', 'REMOVE', 'KEEP', 'AUTO: preferred db and metadata identical.', 'Automated choice. Metadata identical, automatically choose FE23 (Breitenmoser et al. (2014)) over PAGES 2k v2.2.0. conservative replication requirement'], ['407', '3548', 'no figure', 'pages2k_1089', 'FE23_northamerica_usa_mt113', 'PAGES 2k v2.2.0', 'FE23 (Breitenmoser et al. (2014))', 'Yellow Mountain Ridge', 'YellowMountainRidge1-EntireBarkTrees', '45.29999923706055', '45.29999923706055', '-111.30000305175781', '-111.31666564941406', '2500.0', '2500.0', 'Wood', 'Wood', 'ring width', 'ring width', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/NAm-YellowMountainRidge.King.2002.txt', 'https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/northamerica/usa/mt113-noaa.rwl', '470.0-1998.0', '850.0-1998.0', 'REMOVE', 'KEEP', 'AUTO: preferred db and metadata identical.', 'Automated choice. Metadata identical, automatically choose FE23 (Breitenmoser et al. (2014)) over PAGES 2k v2.2.0. conservative replication requirement'], ['415', '4580', 'no figure', 'pages2k_1108', 'iso2k_1060', 'PAGES 2k v2.2.0', 'Iso2k v1.1.2', 'Bermuda south shore', 'Bermuda south shore', '30.64859962463379', '30.64859962463379', '-64.98880004882812', '-64.98880004882812', '-16.0', '-16.0', 'Coral', 'Coral', 'd18O', 'd18O', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-BermudaSouthShore.Goodkin.2008-1.txt', 'https://www.ncdc.noaa.gov/paleo/study/6115', '1782.0-1998.0', '1782.0-1998.0', 'REMOVE', 'KEEP', 'AUTO: IDENTICAL except for URLs and/or geo_siteName.', 'RECORDS IDENTICAL (identical data) except for metadata. Automatically choose #2.'], ['418', '1651', 'no figure', 'pages2k_1116', 'FE23_northamerica_canada_cana170w', 'PAGES 2k v2.2.0', 'FE23 (Breitenmoser et al. (2014))', 'Athabasca, historisch', 'Athabasca,historisch', '51.400001525878906', '51.41666793823242', '-117.30000305175781', '-117.33333587646484', '2000.0', '2000.0', 'Wood', 'Wood', 'ring width', 'ring width', 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/NAm-Athabasca.Schweingruber.1996-1.txt', 'https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/northamerica/canada/cana170w-noaa.rwl', '1072.0-1991.0', '1162.0-1844.0', 'REMOVE', 'KEEP', 'AUTO: preferred db and metadata identical.', 'Automated choice. Metadata identical, automatically choose FE23 (Breitenmoser et al. (2014)) over PAGES 2k v2.2.0. conservative replication requirement']] start with index: 98 ------------------------------------------------------------ Detected MULTIPLE duplicates, including: pages2k_0 3 ['iso2k_296', 'iso2k_298', 'iso2k_299'] pages2k_81 2 ['ch2k_HE08LRA01_76', 'iso2k_1813'] pages2k_225 2 ['FE23_northamerica_usa_nv512', 'FE23_northamerica_usa_nv521'] pages2k_242 2 ['ch2k_LI06FIJ01_582', 'iso2k_353'] pages2k_267 2 ['iso2k_58', 'iso2k_1068'] pages2k_271 2 ['ch2k_FE18RUS01_492', 'iso2k_1861'] pages2k_317 2 ['ch2k_NA09MAL01_84', 'iso2k_1754'] pages2k_385 2 ['ch2k_FE09OGA01_304', 'iso2k_1922'] pages2k_395 2 ['ch2k_CA07FLI01_400', 'iso2k_1057'] pages2k_409 2 ['ch2k_QU96ESV01_422', 'iso2k_218'] pages2k_444 2 ['pages2k_445', 'pages2k_446'] pages2k_462 2 ['ch2k_OS14UCP01_236', 'iso2k_350'] pages2k_468 2 ['pages2k_3550', 'FE23_asia_russ137w'] pages2k_472 2 ['pages2k_474', 'pages2k_477'] pages2k_495 2 ['ch2k_LI06RAR01_12', 'iso2k_1502'] pages2k_500 2 ['ch2k_AS05GUA01_302', 'iso2k_1559'] pages2k_592 2 ['ch2k_LI06RAR02_270', 'iso2k_1500'] pages2k_831 2 ['pages2k_2220', 'FE23_asia_russ127w'] pages2k_893 2 ['pages2k_895', 'pages2k_900'] pages2k_940 3 ['ch2k_DR99ABR01_264', 'ch2k_DR99ABR01_266', 'iso2k_91'] pages2k_1089 2 ['FE23_northamerica_usa_mt112', 'FE23_northamerica_usa_mt113'] pages2k_1147 3 ['ch2k_DA06MAF01_78', 'ch2k_DA06MAF02_104', 'iso2k_1748'] pages2k_1153 2 ['pages2k_1156', 'pages2k_1160'] pages2k_1360 3 ['ch2k_UR00MAI01_22', 'iso2k_94', 'iso2k_98'] pages2k_1488 4 ['pages2k_1628', 'ch2k_NU11PAL01_52', 'iso2k_505', 'iso2k_579'] pages2k_1703 2 ['ch2k_MO06PED01_226', 'iso2k_629'] pages2k_1750 2 ['iso2k_1856', 'sisal_294.0_194'] pages2k_1859 2 ['ch2k_HE10GUA01_244', 'iso2k_1735'] pages2k_1942 2 ['ch2k_ZI04IFR01_26', 'iso2k_257'] pages2k_2042 2 ['ch2k_TU95MAD01_24', 'iso2k_20'] pages2k_2094 2 ['ch2k_TU01DEP01_450', 'iso2k_1201'] pages2k_2146 2 ['pages2k_2149', 'pages2k_2150'] pages2k_2604 2 ['pages2k_2606', 'iso2k_1481'] pages2k_2607 2 ['pages2k_2609', 'pages2k_2612'] pages2k_2752 2 ['pages2k_2755', 'pages2k_2759'] pages2k_2793 2 ['pages2k_2795', 'pages2k_2798'] pages2k_3028 2 ['pages2k_3030', 'pages2k_3033'] pages2k_3068 2 ['ch2k_ZI14IFR02_522', 'ch2k_ZI14IFR02_524'] pages2k_3085 3 ['ch2k_KU00NIN01_150', 'iso2k_1554', 'iso2k_1556'] pages2k_3132 2 ['ch2k_QU06RAB01_144', 'iso2k_1311'] pages2k_3234 2 ['pages2k_3236', 'pages2k_3239'] pages2k_3266 2 ['ch2k_GO12SBV01_396', 'iso2k_870'] pages2k_3352 3 ['ch2k_ZI14TUR01_480', 'ch2k_ZI14TUR01_482', 'iso2k_302'] pages2k_3372 2 ['ch2k_KI04MCV01_366', 'iso2k_155'] pages2k_3554 2 ['ch2k_LI94SEC01_436', 'iso2k_1124'] pages2k_3599 2 ['iso2k_1069', 'iso2k_1660'] ch2k_KU99HOU01_40 2 ['iso2k_786', 'iso2k_788'] ch2k_XI17HAI01_128 2 ['ch2k_XI17HAI01_136', 'iso2k_1762'] ch2k_HE13MIS01_194 2 ['iso2k_211', 'iso2k_213'] ch2k_PF04PBA01_204 2 ['iso2k_1701', 'iso2k_1704'] ch2k_GU99NAU01_314 2 ['iso2k_702', 'iso2k_705'] ch2k_DE13HAI01_424 2 ['ch2k_DE13HAI01_432', 'iso2k_1643'] iso2k_399 2 ['iso2k_806', 'iso2k_811'] iso2k_1107 2 ['iso2k_1817', 'sisal_271.0_174'] PLEASE PAY ATTENTION WHEN MAKING DECISIONS FOR THESE DUPLICATES! The decision process will go through the duplicates on a PAIR-BY-PAIR basis, which is not optimised for multiple duplicates. The multiples will be highlighted throughout the decision process. Should the operator want to go back and revise a previous decision based on the presentation of a new candidate pair, they can manually modify the backup file to alter any previous decisions. ------------------------------------------------------------ 428 4147 98 98 > 99/429,pages2k_1147,ch2k_DA06MAF01_78,0.0,1.0 ==================================================================== === POTENTIAL DUPLICATE 98/429: pages2k_1147+ch2k_DA06MAF01_78 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-Mafia.Damassa.2006.txt === === URL 2: https://www.ncdc.noaa.gov/paleo/study/10808 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) False metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: False
**Decision required for this duplicate pair (see figure above).**
---------------------------------------------------------------------------------------------------------
***ATTENTION*** THIS RECORD IS ASSOCIATED WITH MULTIPLE DUPLICATES! PLEASE PAY SPECIAL ATTENTION WHEN MAKING DECISIONS FOR THIS RECORD!
The potential duplicates also associated with this record are:
............................................................
- Dataset ID : ch2k_DA06MAF02_104
- URL : https://www.ncdc.noaa.gov/paleo/study/10808
............................................................
- Dataset ID : iso2k_1748
- URL : https://www.ncdc.noaa.gov/paleo/study/10808
---------------------------------------------------------------------------------------------------------
saved figure in /home/jupyter-lluecke/dod2k_v2.0/dod2k/figs//dup_detection/all_merged/098_pages2k_1147_ch2k_DA06MAF01_78__428_4147.jpg KEEP RED CROSSES: remove pages2k_1147, keep ch2k_DA06MAF01_78. write decision to backup file 428 4153 98 99 > 100/429,pages2k_1147,ch2k_DA06MAF02_104,0.0,1.0 ==================================================================== === POTENTIAL DUPLICATE 99/429: pages2k_1147+ch2k_DA06MAF02_104 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-Mafia.Damassa.2006.txt === === URL 2: https://www.ncdc.noaa.gov/paleo/study/10808 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) False metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: False
**Decision required for this duplicate pair (see figure above).**
---------------------------------------------------------------------------------------------------------
***ATTENTION*** THIS RECORD IS ASSOCIATED WITH MULTIPLE DUPLICATES! PLEASE PAY SPECIAL ATTENTION WHEN MAKING DECISIONS FOR THIS RECORD!
The potential duplicates also associated with this record are:
............................................................
- Dataset ID : ch2k_DA06MAF01_78
- URL : https://www.ncdc.noaa.gov/paleo/study/10808
............................................................
- Dataset ID : iso2k_1748
- URL : https://www.ncdc.noaa.gov/paleo/study/10808
---------------------------------------------------------------------------------------------------------
saved figure in /home/jupyter-lluecke/dod2k_v2.0/dod2k/figs//dup_detection/all_merged/099_pages2k_1147_ch2k_DA06MAF02_104__428_4153.jpg KEEP RED CROSSES: remove pages2k_1147, keep ch2k_DA06MAF02_104. write decision to backup file 428 4719 98 100 > 101/429,pages2k_1147,iso2k_1748,0.0,1.0 ==================================================================== === POTENTIAL DUPLICATE 100/429: pages2k_1147+iso2k_1748 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-Mafia.Damassa.2006.txt === === URL 2: https://www.ncdc.noaa.gov/paleo/study/10808 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: True correlation_perfect: True RECORDS IDENTICAL (identical data) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove pages2k_1147, keep iso2k_1748. write decision to backup file 431 432 98 101 > 102/429,pages2k_1153,pages2k_1156,0.0,0.9967946398784733 ==================================================================== === POTENTIAL DUPLICATE 101/429: pages2k_1153+pages2k_1156 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-ODP984.Came.2007-1.txt === === URL 2: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-ODP984.Came.2007-1.txt === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: True data_identical: False correlation_perfect: True RECORDS IDENTICAL (perfect correlation). Automatically choose #1. KEEP BLUE CIRCLES: keep pages2k_1153, remove pages2k_1156. write decision to backup file 431 434 98 102 > 103/429,pages2k_1153,pages2k_1160,0.0,0.9969118556506823 ==================================================================== === POTENTIAL DUPLICATE 102/429: pages2k_1153+pages2k_1160 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-ODP984.Came.2007-1.txt === === URL 2: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-ODP984.Came.2007-2.txt === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: False data_identical: False correlation_perfect: True RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #1. KEEP BLUE CIRCLES: keep pages2k_1153, remove pages2k_1160. write decision to backup file 432 434 98 103 > 104/429,pages2k_1156,pages2k_1160,0.0,1.0 ==================================================================== === POTENTIAL DUPLICATE 103/429: pages2k_1156+pages2k_1160 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-ODP984.Came.2007-1.txt === === URL 2: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-ODP984.Came.2007-2.txt === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: False data_identical: False correlation_perfect: True RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #1. KEEP BLUE CIRCLES: keep pages2k_1156, remove pages2k_1160. write decision to backup file 453 3275 98 104 > 105/429,pages2k_1209,FE23_northamerica_usa_co553,4.686190390758007,0.9848164514971338 ==================================================================== === POTENTIAL DUPLICATE 104/429: pages2k_1209+FE23_northamerica_usa_co553 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/NAm-PikePeaks.Harlan.1996-1.txt === === URL 2: https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/northamerica/usa/co553-noaa.rwl === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) False metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: False Automated choice. Metadata identical, automatically choose FE23 (Breitenmoser et al. (2014)) over PAGES 2k v2.2.0. conservative replication requirement KEEP RED CROSSES: remove pages2k_1209, keep FE23_northamerica_usa_co553. write decision to backup file 467 1592 98 105 > 106/429,pages2k_1252,FE23_northamerica_canada_cana096,5.559669351093426,0.9976191515076486 ==================================================================== === POTENTIAL DUPLICATE 105/429: pages2k_1252+FE23_northamerica_canada_cana096 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/NAm-SunwaptaPass.Schweingruber.1996-1.txt === === URL 2: https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/northamerica/canada/cana096-noaa.rwl === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) False metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: False Automated choice. Metadata identical, automatically choose FE23 (Breitenmoser et al. (2014)) over PAGES 2k v2.2.0. conservative replication requirement KEEP RED CROSSES: remove pages2k_1252, keep FE23_northamerica_canada_cana096. write decision to backup file 474 4682 98 106 > 107/429,pages2k_1274,iso2k_1577,0.0,1.0 ==================================================================== === POTENTIAL DUPLICATE 106/429: pages2k_1274+iso2k_1577 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Arc-GRIP.Vinther.2010.txt === === URL 2: https://doi.pangaea.de/10.1594/PANGAEA.786354 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: False lat True lon True elevation False archivetype True paleodata_proxy True sites_identical: True URL_identical: False data_identical: True correlation_perfect: True RECORDS IDENTICAL (identical data) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove pages2k_1274, keep iso2k_1577. write decision to backup file 481 4524 98 107 > 108/429,pages2k_1293,iso2k_821,0.0,0.9999999993592802 ==================================================================== === POTENTIAL DUPLICATE 107/429: pages2k_1293+iso2k_821 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ant-TALDICE.Stenni.2010.txt === === URL 2: https://www.ncdc.noaa.gov/paleo/study/22502 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: True RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove pages2k_1293, keep iso2k_821. write decision to backup file 491 4111 98 108 > 109/429,pages2k_1325,FE23_northamerica_usa_wy030,4.634071826744492,0.9841375865267528 ==================================================================== === POTENTIAL DUPLICATE 108/429: pages2k_1325+FE23_northamerica_usa_wy030 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/NAm-SheepTrail.Brown.2005-1.txt === === URL 2: https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/northamerica/usa/wy030-noaa.rwl === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) False metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: False Automated choice. Metadata identical, automatically choose FE23 (Breitenmoser et al. (2014)) over PAGES 2k v2.2.0. conservative replication requirement KEEP RED CROSSES: remove pages2k_1325, keep FE23_northamerica_usa_wy030. write decision to backup file 502 4126 98 109 > 110/429,pages2k_1360,ch2k_UR00MAI01_22,0.0,1.0 ==================================================================== === POTENTIAL DUPLICATE 109/429: pages2k_1360+ch2k_UR00MAI01_22 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-Maiana.Urban.2000.txt === === URL 2: https://www.ncdc.noaa.gov/paleo/study/1859 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: True RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove pages2k_1360, keep ch2k_UR00MAI01_22. write decision to backup file 502 4362 98 110 > 111/429,pages2k_1360,iso2k_94,7.450067481225913,0.999999993382913 ==================================================================== === POTENTIAL DUPLICATE 110/429: pages2k_1360+iso2k_94 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-Maiana.Urban.2000.txt === === URL 2: https://www.ncdc.noaa.gov/paleo/study/1859 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: False lat True lon True elevation False archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: True RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove pages2k_1360, keep iso2k_94. write decision to backup file 502 4363 98 111 > 112/429,pages2k_1360,iso2k_98,7.450067481225913,0.999999993382913 ==================================================================== === POTENTIAL DUPLICATE 111/429: pages2k_1360+iso2k_98 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-Maiana.Urban.2000.txt === === URL 2: https://www.ncdc.noaa.gov/paleo/study/1859 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: False lat True lon True elevation False archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: True RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove pages2k_1360, keep iso2k_98. write decision to backup file 503 504 98 112 > 113/429,pages2k_1362,pages2k_1365,0.0,0.9992389529344313 ==================================================================== === POTENTIAL DUPLICATE 112/429: pages2k_1362+pages2k_1365 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-GulfofGuinea.Weldeab.2007-1.txt === === URL 2: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-GulfofGuinea.Weldeab.2007-1.txt === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: True data_identical: False correlation_perfect: True RECORDS IDENTICAL (perfect correlation). Automatically choose #1. KEEP BLUE CIRCLES: keep pages2k_1362, remove pages2k_1365. write decision to backup file 505 4689 98 113 > 114/429,pages2k_1370,iso2k_1619,0.0,1.0 ==================================================================== === POTENTIAL DUPLICATE 113/429: pages2k_1370+iso2k_1619 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Arc-NGRIP1.Vinther.2006.txt === === URL 2: https://www.ncdc.noaa.gov/paleo/study/8700 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: True correlation_perfect: True RECORDS IDENTICAL (identical data) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove pages2k_1370, keep iso2k_1619. write decision to backup file 520 1608 98 114 > 115/429,pages2k_1420,FE23_northamerica_canada_cana111,2.210833453585409,0.9185079638747947 ==================================================================== === POTENTIAL DUPLICATE 114/429: pages2k_1420+FE23_northamerica_canada_cana111 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/NAm-VancouverCyprusProvincialPark.Briffa.1996-1.txt === === URL 2: https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/northamerica/canada/cana111-noaa.rwl === True if pot_dup_corrs[i_pot_dups]>=0.98 else False False (len(time_1)==len(time_2)) False metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: False Automated choice. Metadata identical, automatically choose FE23 (Breitenmoser et al. (2014)) over PAGES 2k v2.2.0. conservative replication requirement KEEP RED CROSSES: remove pages2k_1420, keep FE23_northamerica_canada_cana111. write decision to backup file 527 528 98 115 > 116/429,pages2k_1442,pages2k_1444,0.0,1.0 ==================================================================== === POTENTIAL DUPLICATE 115/429: pages2k_1442+pages2k_1444 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-LaurentianFan.Keigwin.2005-1.txt === === URL 2: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-LaurentianFan.Keigwin.2005-2.txt === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: False data_identical: True correlation_perfect: True RECORDS IDENTICAL (identical data) except for metadata. Automatically choose #1. KEEP BLUE CIRCLES: keep pages2k_1442, remove pages2k_1444. write decision to backup file 542 595 98 116 > 117/429,pages2k_1488,pages2k_1628,0.0,0.9999312398195644 ==================================================================== === POTENTIAL DUPLICATE 116/429: pages2k_1488+pages2k_1628 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-Palmyra.Nurhati.2011-1.txt === === URL 2: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-Palmyra.Cobb.2003.txt === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) False metadata_identical: False lat True lon True elevation False archivetype True paleodata_proxy True sites_identical: True URL_identical: False data_identical: False correlation_perfect: False
**Decision required for this duplicate pair (see figure above).**
---------------------------------------------------------------------------------------------------------
***ATTENTION*** THIS RECORD IS ASSOCIATED WITH MULTIPLE DUPLICATES! PLEASE PAY SPECIAL ATTENTION WHEN MAKING DECISIONS FOR THIS RECORD!
The potential duplicates also associated with this record are:
............................................................
- Dataset ID : ch2k_NU11PAL01_52
- URL : https://www.ncdc.noaa.gov/paleo/study/10373
............................................................
- Dataset ID : iso2k_505
- URL : https://www.ncdc.noaa.gov/paleo/study/1875
............................................................
- Dataset ID : iso2k_579
- URL : https://www.ncdc.noaa.gov/paleo/study/10373
---------------------------------------------------------------------------------------------------------
saved figure in /home/jupyter-lluecke/dod2k_v2.0/dod2k/figs//dup_detection/all_merged/116_pages2k_1488_pages2k_1628__542_595.jpg KEEP RED CROSSES: remove pages2k_1488, keep pages2k_1628. write decision to backup file 542 4138 98 117 > 118/429,pages2k_1488,ch2k_NU11PAL01_52,0.4697858835846662,0.9992710430546085 ==================================================================== === POTENTIAL DUPLICATE 117/429: pages2k_1488+ch2k_NU11PAL01_52 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-Palmyra.Nurhati.2011-1.txt === === URL 2: https://www.ncdc.noaa.gov/paleo/study/10373 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: True RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove pages2k_1488, keep ch2k_NU11PAL01_52. write decision to backup file 542 4456 98 118 > 119/429,pages2k_1488,iso2k_505,0.0,0.9976024758754877 ==================================================================== === POTENTIAL DUPLICATE 118/429: pages2k_1488+iso2k_505 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-Palmyra.Nurhati.2011-1.txt === === URL 2: https://www.ncdc.noaa.gov/paleo/study/1875 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) False metadata_identical: False lat True lon True elevation False archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: False
**Decision required for this duplicate pair (see figure above).**
---------------------------------------------------------------------------------------------------------
***ATTENTION*** THIS RECORD IS ASSOCIATED WITH MULTIPLE DUPLICATES! PLEASE PAY SPECIAL ATTENTION WHEN MAKING DECISIONS FOR THIS RECORD!
The potential duplicates also associated with this record are:
............................................................
- Dataset ID : pages2k_1628
- URL : https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-Palmyra.Cobb.2003.txt
............................................................
- Dataset ID : ch2k_NU11PAL01_52
- URL : https://www.ncdc.noaa.gov/paleo/study/10373
............................................................
- Dataset ID : iso2k_579
- URL : https://www.ncdc.noaa.gov/paleo/study/10373
---------------------------------------------------------------------------------------------------------
saved figure in /home/jupyter-lluecke/dod2k_v2.0/dod2k/figs//dup_detection/all_merged/118_pages2k_1488_iso2k_505__542_4456.jpg KEEP BLUE CIRCLES: keep pages2k_1488, remove iso2k_505. write decision to backup file 542 4482 98 119 > 120/429,pages2k_1488,iso2k_579,0.0,1.0 ==================================================================== === POTENTIAL DUPLICATE 119/429: pages2k_1488+iso2k_579 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-Palmyra.Nurhati.2011-1.txt === === URL 2: https://www.ncdc.noaa.gov/paleo/study/10373 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: False data_identical: True correlation_perfect: True RECORDS IDENTICAL (identical data) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove pages2k_1488, keep iso2k_579. write decision to backup file 543 4139 98 120 > 121/429,pages2k_1490,ch2k_NU11PAL01_54,0.4697858835846662,0.999992275602027 ==================================================================== === POTENTIAL DUPLICATE 120/429: pages2k_1490+ch2k_NU11PAL01_54 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-Palmyra.Nurhati.2011-1.txt === === URL 2: https://www.ncdc.noaa.gov/paleo/study/10373 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: True RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove pages2k_1490, keep ch2k_NU11PAL01_54. write decision to backup file 544 4481 98 121 > 122/429,pages2k_1491,iso2k_575,0.0,1.0 ==================================================================== === POTENTIAL DUPLICATE 121/429: pages2k_1491+iso2k_575 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-Palmyra.Nurhati.2011-2.txt === === URL 2: https://www.ncdc.noaa.gov/paleo/study/10373 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: False data_identical: True correlation_perfect: True RECORDS IDENTICAL (identical data) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove pages2k_1491, keep iso2k_575. write decision to backup file 547 4761 98 122 > 123/429,pages2k_1497,iso2k_1885,0.0,1.0 ==================================================================== === POTENTIAL DUPLICATE 122/429: pages2k_1497+iso2k_1885 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/SAm-QuelccayaIceCap.Thompson.2013.txt === === URL 2: https://www.ncdc.noaa.gov/paleo/study/14174 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: False data_identical: True correlation_perfect: True RECORDS IDENTICAL (identical data) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove pages2k_1497, keep iso2k_1885. write decision to backup file 550 552 98 123 > 124/429,pages2k_1515,pages2k_1519,0.0,1.0 ==================================================================== === POTENTIAL DUPLICATE 123/429: pages2k_1515+pages2k_1519 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-SouthChinaSea.Zhao.2006-1.txt === === URL 2: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-SouthChinaSea.Zhao.2006-2.txt === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: False data_identical: True correlation_perfect: True RECORDS IDENTICAL (identical data) except for metadata. Automatically choose #1. KEEP BLUE CIRCLES: keep pages2k_1515, remove pages2k_1519. write decision to backup file 553 554 98 124 > 125/429,pages2k_1520,pages2k_1522,0.0,1.0 ==================================================================== === POTENTIAL DUPLICATE 124/429: pages2k_1520+pages2k_1522 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-SubTropicalEasternNorthAtlantic.deMenocal.2000-1.txt === === URL 2: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-SubTropicalEasternNorthAtlantic.deMenocal.2000-2.txt === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) False metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: False data_identical: False correlation_perfect: False
**Decision required for this duplicate pair (see figure above).**
saved figure in /home/jupyter-lluecke/dod2k_v2.0/dod2k/figs//dup_detection/all_merged/124_pages2k_1520_pages2k_1522__553_554.jpg KEEP RED CROSSES: remove pages2k_1520, keep pages2k_1522. write decision to backup file 564 4396 98 125 > 126/429,pages2k_1547,iso2k_259,0.0,1.0 ==================================================================== === POTENTIAL DUPLICATE 125/429: pages2k_1547+iso2k_259 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-Kiritimati.Evans.1998.txt === === URL 2: https://www.ncdc.noaa.gov/paleo/study/1847 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: False lat True lon True elevation False archivetype True paleodata_proxy True sites_identical: True URL_identical: False data_identical: True correlation_perfect: True RECORDS IDENTICAL (identical data) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove pages2k_1547, keep iso2k_259. write decision to backup file 573 1712 98 126 > 127/429,pages2k_1566,FE23_northamerica_canada_cana231,4.390485654920984,0.9977556148240536 ==================================================================== === POTENTIAL DUPLICATE 126/429: pages2k_1566+FE23_northamerica_canada_cana231 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/NAm-ParkMountain.Wilson.2005-1.txt === === URL 2: https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/northamerica/canada/cana231-noaa.rwl === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) False metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: False Automated choice. Metadata identical, automatically choose FE23 (Breitenmoser et al. (2014)) over PAGES 2k v2.2.0. conservative replication requirement KEEP RED CROSSES: remove pages2k_1566, keep FE23_northamerica_canada_cana231. write decision to backup file 585 3154 98 127 > 128/429,pages2k_1605,FE23_northamerica_usa_ca606,3.985226443144479,0.9693885470026882 ==================================================================== === POTENTIAL DUPLICATE 127/429: pages2k_1605+FE23_northamerica_usa_ca606 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/NAm-SpillwayLakeYosemiteNationalPark.King.2000.txt === === URL 2: https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/northamerica/usa/ca606-noaa.rwl === True if pot_dup_corrs[i_pot_dups]>=0.98 else False False (len(time_1)==len(time_2)) False metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: False Automated choice. Metadata identical, automatically choose FE23 (Breitenmoser et al. (2014)) over PAGES 2k v2.2.0. conservative replication requirement KEEP RED CROSSES: remove pages2k_1605, keep FE23_northamerica_usa_ca606. write decision to backup file 590 592 98 128 > 129/429,pages2k_1619,pages2k_1623,0.0,1.0 ==================================================================== === POTENTIAL DUPLICATE 128/429: pages2k_1619+pages2k_1623 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-JacafFjord.Seplveda.2009-1.txt === === URL 2: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-JacafFjord.Seplveda.2009-2.txt === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) False metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: False data_identical: False correlation_perfect: False
**Decision required for this duplicate pair (see figure above).**
saved figure in /home/jupyter-lluecke/dod2k_v2.0/dod2k/figs//dup_detection/all_merged/128_pages2k_1619_pages2k_1623__590_592.jpg KEEP RED CROSSES: remove pages2k_1619, keep pages2k_1623. write decision to backup file 595 4138 98 129 > 130/429,pages2k_1628,ch2k_NU11PAL01_52,0.4697858835846662,0.9999312398195646 ==================================================================== === POTENTIAL DUPLICATE 129/429: pages2k_1628+ch2k_NU11PAL01_52 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-Palmyra.Cobb.2003.txt === === URL 2: https://www.ncdc.noaa.gov/paleo/study/10373 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) False metadata_identical: False lat True lon True elevation False archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: False
**Decision required for this duplicate pair (see figure above).**
---------------------------------------------------------------------------------------------------------
***ATTENTION*** THIS RECORD IS ASSOCIATED WITH MULTIPLE DUPLICATES! PLEASE PAY SPECIAL ATTENTION WHEN MAKING DECISIONS FOR THIS RECORD!
The potential duplicates also associated with this record are:
- pages2k_1488
............................................................
- Dataset ID : iso2k_505
- URL : https://www.ncdc.noaa.gov/paleo/study/1875
............................................................
- Dataset ID : iso2k_579
- URL : https://www.ncdc.noaa.gov/paleo/study/10373
---------------------------------------------------------------------------------------------------------
saved figure in /home/jupyter-lluecke/dod2k_v2.0/dod2k/figs//dup_detection/all_merged/129_pages2k_1628_ch2k_NU11PAL01_52__595_4138.jpg KEEP BLUE CIRCLES: keep pages2k_1628, remove ch2k_NU11PAL01_52. write decision to backup file 595 4456 98 130 > 131/429,pages2k_1628,iso2k_505,0.0,0.9973624471178902 ==================================================================== === POTENTIAL DUPLICATE 130/429: pages2k_1628+iso2k_505 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-Palmyra.Cobb.2003.txt === === URL 2: https://www.ncdc.noaa.gov/paleo/study/1875 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) False metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: False
**Decision required for this duplicate pair (see figure above).**
---------------------------------------------------------------------------------------------------------
***ATTENTION*** THIS RECORD IS ASSOCIATED WITH MULTIPLE DUPLICATES! PLEASE PAY SPECIAL ATTENTION WHEN MAKING DECISIONS FOR THIS RECORD!
The potential duplicates also associated with this record are:
- pages2k_1488
............................................................
- Dataset ID : ch2k_NU11PAL01_52
- URL : https://www.ncdc.noaa.gov/paleo/study/10373
............................................................
- Dataset ID : iso2k_579
- URL : https://www.ncdc.noaa.gov/paleo/study/10373
---------------------------------------------------------------------------------------------------------
saved figure in /home/jupyter-lluecke/dod2k_v2.0/dod2k/figs//dup_detection/all_merged/130_pages2k_1628_iso2k_505__595_4456.jpg KEEP BLUE CIRCLES: keep pages2k_1628, remove iso2k_505. write decision to backup file 595 4482 98 131 > 132/429,pages2k_1628,iso2k_579,0.0,0.9999312398195646 ==================================================================== === POTENTIAL DUPLICATE 131/429: pages2k_1628+iso2k_579 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-Palmyra.Cobb.2003.txt === === URL 2: https://www.ncdc.noaa.gov/paleo/study/10373 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) False metadata_identical: False lat True lon True elevation False archivetype True paleodata_proxy True sites_identical: True URL_identical: False data_identical: False correlation_perfect: False
**Decision required for this duplicate pair (see figure above).**
---------------------------------------------------------------------------------------------------------
***ATTENTION*** THIS RECORD IS ASSOCIATED WITH MULTIPLE DUPLICATES! PLEASE PAY SPECIAL ATTENTION WHEN MAKING DECISIONS FOR THIS RECORD!
The potential duplicates also associated with this record are:
- pages2k_1488
............................................................
- Dataset ID : ch2k_NU11PAL01_52
- URL : https://www.ncdc.noaa.gov/paleo/study/10373
............................................................
- Dataset ID : iso2k_505
- URL : https://www.ncdc.noaa.gov/paleo/study/1875
---------------------------------------------------------------------------------------------------------
saved figure in /home/jupyter-lluecke/dod2k_v2.0/dod2k/figs//dup_detection/all_merged/131_pages2k_1628_iso2k_579__595_4482.jpg KEEP BLUE CIRCLES: keep pages2k_1628, remove iso2k_579. write decision to backup file 597 4030 98 132 > 133/429,pages2k_1636,FE23_northamerica_usa_wa081,6.762315219649745,0.9980630247518526 ==================================================================== === POTENTIAL DUPLICATE 132/429: pages2k_1636+FE23_northamerica_usa_wa081 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/NAm-MtStHelens.Briffa.1996-1.txt === === URL 2: https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/northamerica/usa/wa081-noaa.rwl === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) False metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: False Automated choice. Metadata identical, automatically choose FE23 (Breitenmoser et al. (2014)) over PAGES 2k v2.2.0. conservative replication requirement KEEP RED CROSSES: remove pages2k_1636, keep FE23_northamerica_usa_wa081. write decision to backup file 614 615 98 133 > 134/429,pages2k_1686,pages2k_1688,0.0,1.0 ==================================================================== === POTENTIAL DUPLICATE 133/429: pages2k_1686+pages2k_1688 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-ArabianSea.Doose-Rolinski.2001-1.txt === === URL 2: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-ArabianSea.Doose-Rolinski.2001-2.txt === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) False metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: False data_identical: False correlation_perfect: False
**Decision required for this duplicate pair (see figure above).**
saved figure in /home/jupyter-lluecke/dod2k_v2.0/dod2k/figs//dup_detection/all_merged/133_pages2k_1686_pages2k_1688__614_615.jpg KEEP BOTH: keep pages2k_1686, keep pages2k_1688. write decision to backup file 617 2421 98 134 > 135/429,pages2k_1692,FE23_asia_mong012,1.040938228538874,0.981578554195544 ==================================================================== === POTENTIAL DUPLICATE 134/429: pages2k_1692+FE23_asia_mong012 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Asi-MONG012.Jacoby.2006.txt === === URL 2: https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/asia/mong012-noaa.rwl === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) False metadata_identical: False lat True lon True elevation False archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: False
**Decision required for this duplicate pair (see figure above).**
saved figure in /home/jupyter-lluecke/dod2k_v2.0/dod2k/figs//dup_detection/all_merged/134_pages2k_1692_FE23_asia_mong012__617_2421.jpg KEEP RED CROSSES: remove pages2k_1692, keep FE23_asia_mong012. write decision to backup file 619 4204 98 135 > 136/429,pages2k_1703,ch2k_MO06PED01_226,0.1809429642245807,1.0 ==================================================================== === POTENTIAL DUPLICATE 135/429: pages2k_1703+ch2k_MO06PED01_226 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-PedradeLume-CapeVerdeIslands.Moses.2006.txt === === URL 2: https://www.ncdc.noaa.gov/paleo/study/6184 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: True correlation_perfect: True RECORDS IDENTICAL (identical data) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove pages2k_1703, keep ch2k_MO06PED01_226. write decision to backup file 619 4490 98 136 > 137/429,pages2k_1703,iso2k_629,1.1119762877766939,1.0 ==================================================================== === POTENTIAL DUPLICATE 136/429: pages2k_1703+iso2k_629 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-PedradeLume-CapeVerdeIslands.Moses.2006.txt === === URL 2: https://www.ncdc.noaa.gov/paleo/study/6184 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: False data_identical: False correlation_perfect: True RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove pages2k_1703, keep iso2k_629. write decision to backup file 624 4504 98 137 > 138/429,pages2k_1712,iso2k_715,1.1553943531955169,1.0 ==================================================================== === POTENTIAL DUPLICATE 137/429: pages2k_1712+iso2k_715 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-BiscayneBay.Swart.1996.txt === === URL 2: https://www.ncdc.noaa.gov/paleo/study/1885 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: True correlation_perfect: True RECORDS IDENTICAL (identical data) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove pages2k_1712, keep iso2k_715. write decision to backup file 628 4683 98 138 > 139/429,pages2k_1720,iso2k_1579,0.0,0.9999999995576067 ==================================================================== === POTENTIAL DUPLICATE 138/429: pages2k_1720+iso2k_1579 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Arc-PrinceOfWales.Kinnard.2011.txt === === URL 2: https://www.ncdc.noaa.gov/paleo/study/22542 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: True RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove pages2k_1720, keep iso2k_1579. write decision to backup file 635 4053 98 139 > 140/429,pages2k_1741,FE23_northamerica_usa_wa104,3.706446234061719,0.9933646920496347 ==================================================================== === POTENTIAL DUPLICATE 139/429: pages2k_1741+FE23_northamerica_usa_wa104 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/NAm-MountAdamsLow.Peterson.1994.txt === === URL 2: https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/northamerica/usa/wa104-noaa.rwl === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) False metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: False Automated choice. Metadata identical, automatically choose FE23 (Breitenmoser et al. (2014)) over PAGES 2k v2.2.0. conservative replication requirement KEEP RED CROSSES: remove pages2k_1741, keep FE23_northamerica_usa_wa104. write decision to backup file 638 4752 98 140 > 141/429,pages2k_1750,iso2k_1856,1.818967049399386,1.0 ==================================================================== === POTENTIAL DUPLICATE 140/429: pages2k_1750+iso2k_1856 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/NAm-OregonCaves.Ersek.2012.txt === === URL 2: https://www.ncdc.noaa.gov/paleo/study/13543 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: False lat True lon True elevation False archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: True correlation_perfect: True RECORDS IDENTICAL (identical data) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove pages2k_1750, keep iso2k_1856. write decision to backup file 638 4968 98 141 > 142/429,pages2k_1750,sisal_294.0_194,1.8290844982587682,1.0 ==================================================================== === POTENTIAL DUPLICATE 141/429: pages2k_1750+sisal_294.0_194 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/NAm-OregonCaves.Ersek.2012.txt === === URL 2: ['10.1038/ncomms2222'] === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: False lat True lon True elevation False archivetype False paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: True RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove pages2k_1750, keep sisal_294.0_194. write decision to backup file 644 4190 98 142 > 143/429,pages2k_1771,ch2k_TU01LAI01_192,0.033845071807101175,0.9999999999999999 ==================================================================== === POTENTIAL DUPLICATE 142/429: pages2k_1771+ch2k_TU01LAI01_192 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-LaingIslandPapuaNewGuinea.Tudhope.2001.txt === === URL 2: https://www.ncdc.noaa.gov/paleo/study/1866 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: False data_identical: False correlation_perfect: True RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove pages2k_1771, keep ch2k_TU01LAI01_192. write decision to backup file 656 3441 98 143 > 144/429,pages2k_1804,FE23_northamerica_usa_me010,4.545169326471252,0.9575116284079646 ==================================================================== === POTENTIAL DUPLICATE 143/429: pages2k_1804+FE23_northamerica_usa_me010 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/NAm-ElephantMountain.Conkey.1994-1.txt === === URL 2: https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/northamerica/usa/me010-noaa.rwl === True if pot_dup_corrs[i_pot_dups]>=0.98 else False False (len(time_1)==len(time_2)) False metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: False Automated choice. Metadata identical, automatically choose FE23 (Breitenmoser et al. (2014)) over PAGES 2k v2.2.0. conservative replication requirement KEEP RED CROSSES: remove pages2k_1804, keep FE23_northamerica_usa_me010. write decision to backup file 673 4212 98 144 > 145/429,pages2k_1859,ch2k_HE10GUA01_244,0.0,0.9999993553490845 ==================================================================== === POTENTIAL DUPLICATE 144/429: pages2k_1859+ch2k_HE10GUA01_244 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-Guadeloupe.Steinhilber.2010.txt === === URL 2: https://www.ncdc.noaa.gov/paleo/study/12893 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: True RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove pages2k_1859, keep ch2k_HE10GUA01_244. write decision to backup file 673 4715 98 145 > 146/429,pages2k_1859,iso2k_1735,0.0,0.9999999953474713 ==================================================================== === POTENTIAL DUPLICATE 145/429: pages2k_1859+iso2k_1735 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-Guadeloupe.Steinhilber.2010.txt === === URL 2: https://www.ncdc.noaa.gov/paleo/study/12893 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: False data_identical: False correlation_perfect: True RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove pages2k_1859, keep iso2k_1735. write decision to backup file 674 4213 98 146 > 147/429,pages2k_1861,ch2k_HE10GUA01_246,0.0,0.9999903684972697 ==================================================================== === POTENTIAL DUPLICATE 146/429: pages2k_1861+ch2k_HE10GUA01_246 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-Guadeloupe.Steinhilber.2010.txt === === URL 2: https://www.ncdc.noaa.gov/paleo/study/12893 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: True RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove pages2k_1861, keep ch2k_HE10GUA01_246. write decision to backup file 680 2823 98 147 > 148/429,pages2k_1880,FE23_northamerica_usa_ak060,2.9785411732486518,0.9777961919437125 ==================================================================== === POTENTIAL DUPLICATE 147/429: pages2k_1880+FE23_northamerica_usa_ak060 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/NAm-BurntOver.DArrigo.2005.txt === === URL 2: https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/northamerica/usa/ak060-noaa.rwl === True if pot_dup_corrs[i_pot_dups]>=0.98 else False False (len(time_1)==len(time_2)) False metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: False Automated choice. Metadata identical, automatically choose FE23 (Breitenmoser et al. (2014)) over PAGES 2k v2.2.0. conservative replication requirement KEEP RED CROSSES: remove pages2k_1880, keep FE23_northamerica_usa_ak060. write decision to backup file 684 685 98 148 > 149/429,pages2k_1891,pages2k_1893,0.0,0.9999999999999998 ==================================================================== === POTENTIAL DUPLICATE 148/429: pages2k_1891+pages2k_1893 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-CapeGhir.Doose-Rolinski.2007-1.txt === === URL 2: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-CapeGhir.Doose-Rolinski.2007-2.txt === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) False metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: False data_identical: False correlation_perfect: False
**Decision required for this duplicate pair (see figure above).**
saved figure in /home/jupyter-lluecke/dod2k_v2.0/dod2k/figs//dup_detection/all_merged/148_pages2k_1891_pages2k_1893__684_685.jpg KEEP RED CROSSES: remove pages2k_1891, keep pages2k_1893. write decision to backup file 695 4365 98 149 > 150/429,pages2k_1918,iso2k_102,0.0,1.0 ==================================================================== === POTENTIAL DUPLICATE 149/429: pages2k_1918+iso2k_102 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ant-SipleStation.Mosley-Thompson.1990.txt === === URL 2: www.ncdc.noaa.gov/paleo-search/study/27330 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: False data_identical: True correlation_perfect: True RECORDS IDENTICAL (identical data) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove pages2k_1918, keep iso2k_102. write decision to backup file 696 697 98 150 > 151/429,pages2k_1920,pages2k_1923,0.0,0.9983731192234753 ==================================================================== === POTENTIAL DUPLICATE 150/429: pages2k_1920+pages2k_1923 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-PigmyBasin.Richey.2015-1.txt === === URL 2: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-PigmyBasin.Richey.2015-1.txt === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: True data_identical: False correlation_perfect: True RECORDS IDENTICAL (perfect correlation). Automatically choose #1. KEEP BLUE CIRCLES: keep pages2k_1920, remove pages2k_1923. write decision to backup file 700 701 98 151 > 152/429,pages2k_1932,pages2k_1934,0.0,1.0 ==================================================================== === POTENTIAL DUPLICATE 151/429: pages2k_1932+pages2k_1934 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-MD95-2011.Grimalt.2002-1.txt === === URL 2: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-MD95-2011.Grimalt.2002-2.txt === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) False metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: False data_identical: False correlation_perfect: False
**Decision required for this duplicate pair (see figure above).**
saved figure in /home/jupyter-lluecke/dod2k_v2.0/dod2k/figs//dup_detection/all_merged/151_pages2k_1932_pages2k_1934__700_701.jpg KEEP RED CROSSES: remove pages2k_1932, keep pages2k_1934. write decision to backup file 705 4128 98 152 > 153/429,pages2k_1942,ch2k_ZI04IFR01_26,0.0,1.0 ==================================================================== === POTENTIAL DUPLICATE 152/429: pages2k_1942+ch2k_ZI04IFR01_26 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-Ifaty4.Zinke.2004.txt === === URL 2: https://www.ncdc.noaa.gov/paleo/study/1897 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: True RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove pages2k_1942, keep ch2k_ZI04IFR01_26. write decision to backup file 705 4395 98 153 > 154/429,pages2k_1942,iso2k_257,0.0,1.0 ==================================================================== === POTENTIAL DUPLICATE 153/429: pages2k_1942+iso2k_257 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-Ifaty4.Zinke.2004.txt === === URL 2: https://www.ncdc.noaa.gov/paleo/study/16438 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: True correlation_perfect: True RECORDS IDENTICAL (identical data) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove pages2k_1942, keep iso2k_257. write decision to backup file 716 717 98 154 > 155/429,pages2k_1972,pages2k_1973,0.0,0.999919760317777 ==================================================================== === POTENTIAL DUPLICATE 154/429: pages2k_1972+pages2k_1973 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Afr-LakeMalawi.Powers.2011.txt === === URL 2: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Afr-LakeMalawi.Powers.2011.txt === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: True data_identical: False correlation_perfect: True RECORDS IDENTICAL (perfect correlation). Automatically choose #1. KEEP BLUE CIRCLES: keep pages2k_1972, remove pages2k_1973. write decision to backup file 718 720 98 155 > 156/429,pages2k_1976,pages2k_1980,0.0,1.0 ==================================================================== === POTENTIAL DUPLICATE 155/429: pages2k_1976+pages2k_1980 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-MinorcaContourite.Moreno.2012-1.txt === === URL 2: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-MinorcaContourite.Moreno.2012-2.txt === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) False metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: False data_identical: False correlation_perfect: False
**Decision required for this duplicate pair (see figure above).**
saved figure in /home/jupyter-lluecke/dod2k_v2.0/dod2k/figs//dup_detection/all_merged/155_pages2k_1976_pages2k_1980__718_720.jpg KEEP RED CROSSES: remove pages2k_1976, keep pages2k_1980. write decision to backup file 719 721 98 156 > 157/429,pages2k_1978,pages2k_1983,0.0,1.0 ==================================================================== === POTENTIAL DUPLICATE 156/429: pages2k_1978+pages2k_1983 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-MinorcaContourite.Moreno.2012-1.txt === === URL 2: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-MinorcaContourite.Moreno.2012-2.txt === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) False metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: False data_identical: False correlation_perfect: False
**Decision required for this duplicate pair (see figure above).**
saved figure in /home/jupyter-lluecke/dod2k_v2.0/dod2k/figs//dup_detection/all_merged/156_pages2k_1978_pages2k_1983__719_721.jpg KEEP RED CROSSES: remove pages2k_1978, keep pages2k_1983. write decision to backup file 722 4622 98 157 > 158/429,pages2k_1985,iso2k_1294,0.022267136932497017,1.0 ==================================================================== === POTENTIAL DUPLICATE 157/429: pages2k_1985+iso2k_1294 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Arc-NGTB16.Schwager.1998.txt === === URL 2: https://doi.pangaea.de/10.1594/PANGAEA.849161 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: True correlation_perfect: True RECORDS IDENTICAL (identical data) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove pages2k_1985, keep iso2k_1294. write decision to backup file 724 725 98 158 > 159/429,pages2k_1989,pages2k_1991,0.0,0.9999999999990443 ==================================================================== === POTENTIAL DUPLICATE 158/429: pages2k_1989+pages2k_1991 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Afr-LakeTanganyika.Tierney.2010.txt === === URL 2: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Afr-LakeTanganyika.Tierney.2010.txt === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: True data_identical: False correlation_perfect: True RECORDS IDENTICAL (perfect correlation). Automatically choose #1. KEEP BLUE CIRCLES: keep pages2k_1989, remove pages2k_1991. write decision to backup file 726 4216 98 159 > 160/429,pages2k_1994,ch2k_DE12ANC01_258,0.8847115797978425,0.9988073192203925 ==================================================================== === POTENTIAL DUPLICATE 159/429: pages2k_1994+ch2k_DE12ANC01_258 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-AmedeeIsland.DeLong.2012.txt === === URL 2: https://www.ncdc.noaa.gov/paleo/study/13035 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: True RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove pages2k_1994, keep ch2k_DE12ANC01_258. write decision to backup file 732 1593 98 160 > 161/429,pages2k_2013,FE23_northamerica_canada_cana097,5.676742833608837,0.9782537218847712 ==================================================================== === POTENTIAL DUPLICATE 160/429: pages2k_2013+FE23_northamerica_canada_cana097 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/NAm-PeytoLake.Schweingruber.1996-1.txt === === URL 2: https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/northamerica/canada/cana097-noaa.rwl === True if pot_dup_corrs[i_pot_dups]>=0.98 else False False (len(time_1)==len(time_2)) False metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: False Automated choice. Metadata identical, automatically choose FE23 (Breitenmoser et al. (2014)) over PAGES 2k v2.2.0. conservative replication requirement KEEP RED CROSSES: remove pages2k_2013, keep FE23_northamerica_canada_cana097. write decision to backup file 742 4127 98 161 > 162/429,pages2k_2042,ch2k_TU95MAD01_24,0.0,0.9999999999999999 ==================================================================== === POTENTIAL DUPLICATE 161/429: pages2k_2042+ch2k_TU95MAD01_24 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-MadangLagoonPapuaNewGuinea.Tudhope.1995.txt === === URL 2: https://www.ncdc.noaa.gov/paleo/study/1844 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: False lat True lon True elevation False archivetype True paleodata_proxy True sites_identical: True URL_identical: False data_identical: False correlation_perfect: True RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove pages2k_2042, keep ch2k_TU95MAD01_24. write decision to backup file 742 4342 98 162 > 163/429,pages2k_2042,iso2k_20,0.0,0.9999999999999999 ==================================================================== === POTENTIAL DUPLICATE 162/429: pages2k_2042+iso2k_20 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-MadangLagoonPapuaNewGuinea.Tudhope.1995.txt === === URL 2: https://www.ncdc.noaa.gov/paleo/study/1844 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: True correlation_perfect: True RECORDS IDENTICAL (identical data) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove pages2k_2042, keep iso2k_20. write decision to backup file 750 2821 98 163 > 164/429,pages2k_2059,FE23_northamerica_usa_ak058,2.0092840563554346,0.9938599924899512 ==================================================================== === POTENTIAL DUPLICATE 163/429: pages2k_2059+FE23_northamerica_usa_ak058 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/NAm-AlmondButterUpper.DArrigo.2005.txt === === URL 2: https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/northamerica/usa/ak058-noaa.rwl === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) False metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: False Automated choice. Metadata identical, automatically choose FE23 (Breitenmoser et al. (2014)) over PAGES 2k v2.2.0. conservative replication requirement KEEP RED CROSSES: remove pages2k_2059, keep FE23_northamerica_usa_ak058. write decision to backup file 758 1517 98 164 > 165/429,pages2k_2085,FE23_northamerica_canada_cana002,3.8321240746850656,0.980212847340203 ==================================================================== === POTENTIAL DUPLICATE 164/429: pages2k_2085+FE23_northamerica_canada_cana002 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/NAm-FortChimo.Fritts.1981.txt === === URL 2: https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/northamerica/canada/cana002-noaa.rwl === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) False metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: False Automated choice. Metadata identical, automatically choose FE23 (Breitenmoser et al. (2014)) over PAGES 2k v2.2.0. conservative replication requirement KEEP RED CROSSES: remove pages2k_2085, keep FE23_northamerica_canada_cana002. write decision to backup file 761 4290 98 165 > 166/429,pages2k_2094,ch2k_TU01DEP01_450,0.047479212299464564,1.0 ==================================================================== === POTENTIAL DUPLICATE 165/429: pages2k_2094+ch2k_TU01DEP01_450 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-MadangLagoonPapuaNewGuinea.Kuhnert.2001.txt === === URL 2: https://www.ncdc.noaa.gov/paleo/study/1866 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: False data_identical: False correlation_perfect: True RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove pages2k_2094, keep ch2k_TU01DEP01_450. write decision to backup file 761 4602 98 166 > 167/429,pages2k_2094,iso2k_1201,0.047479212299464564,1.0 ==================================================================== === POTENTIAL DUPLICATE 166/429: pages2k_2094+iso2k_1201 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-MadangLagoonPapuaNewGuinea.Kuhnert.2001.txt === === URL 2: https://www.ncdc.noaa.gov/paleo/study/1866 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: True correlation_perfect: True RECORDS IDENTICAL (identical data) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove pages2k_2094, keep iso2k_1201. write decision to backup file 763 765 98 167 > 168/429,pages2k_2098,pages2k_2103,0.0,0.990390473253425 ==================================================================== === POTENTIAL DUPLICATE 167/429: pages2k_2098+pages2k_2103 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-CariacoBasin.Lea.2003-1.txt === === URL 2: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-CariacoBasin.Lea.2003-2.txt === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) False metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: False data_identical: False correlation_perfect: False
**Decision required for this duplicate pair (see figure above).**
saved figure in /home/jupyter-lluecke/dod2k_v2.0/dod2k/figs//dup_detection/all_merged/167_pages2k_2098_pages2k_2103__763_765.jpg KEEP BOTH: keep pages2k_2098, keep pages2k_2103. write decision to backup file 768 3276 98 168 > 169/429,pages2k_2110,FE23_northamerica_usa_co554,3.978640208073092,0.9663565642909764 ==================================================================== === POTENTIAL DUPLICATE 168/429: pages2k_2110+FE23_northamerica_usa_co554 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/NAm-CottonwoodPass.Briffa.1996-1.txt === === URL 2: https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/northamerica/usa/co554-noaa.rwl === True if pot_dup_corrs[i_pot_dups]>=0.98 else False False (len(time_1)==len(time_2)) False metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: False Automated choice. Metadata identical, automatically choose FE23 (Breitenmoser et al. (2014)) over PAGES 2k v2.2.0. conservative replication requirement KEEP RED CROSSES: remove pages2k_2110, keep FE23_northamerica_usa_co554. write decision to backup file 782 784 98 169 > 170/429,pages2k_2146,pages2k_2149,0.0,0.9676859064640569 ==================================================================== === POTENTIAL DUPLICATE 169/429: pages2k_2146+pages2k_2149 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/SAm-CentralAndes6.Villalba.2014.txt === === URL 2: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/SAm-CentralAndes6.Villalba.2014.txt === True if pot_dup_corrs[i_pot_dups]>=0.98 else False False (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: True data_identical: False correlation_perfect: False
**Decision required for this duplicate pair (see figure above).**
---------------------------------------------------------------------------------------------------------
***ATTENTION*** THIS RECORD IS ASSOCIATED WITH MULTIPLE DUPLICATES! PLEASE PAY SPECIAL ATTENTION WHEN MAKING DECISIONS FOR THIS RECORD!
The potential duplicates also associated with this record are:
............................................................
- Dataset ID : pages2k_2150
- URL : https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/SAm-CentralAndes6.Villalba.2014.txt
---------------------------------------------------------------------------------------------------------
saved figure in /home/jupyter-lluecke/dod2k_v2.0/dod2k/figs//dup_detection/all_merged/169_pages2k_2146_pages2k_2149__782_784.jpg KEEP BOTH: keep pages2k_2146, keep pages2k_2149. write decision to backup file 782 785 98 170 > 171/429,pages2k_2146,pages2k_2150,0.0,0.9832956643776358 ==================================================================== === POTENTIAL DUPLICATE 170/429: pages2k_2146+pages2k_2150 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/SAm-CentralAndes6.Villalba.2014.txt === === URL 2: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/SAm-CentralAndes6.Villalba.2014.txt === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: True data_identical: False correlation_perfect: True RECORDS IDENTICAL (perfect correlation). Automatically choose #1. KEEP BLUE CIRCLES: keep pages2k_2146, remove pages2k_2150. write decision to backup file 784 785 98 171 > 172/429,pages2k_2149,pages2k_2150,0.0,0.9910167365890014 ==================================================================== === POTENTIAL DUPLICATE 171/429: pages2k_2149+pages2k_2150 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/SAm-CentralAndes6.Villalba.2014.txt === === URL 2: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/SAm-CentralAndes6.Villalba.2014.txt === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: True data_identical: False correlation_perfect: True RECORDS IDENTICAL (perfect correlation). Automatically choose #1. KEEP BLUE CIRCLES: keep pages2k_2149, remove pages2k_2150. write decision to backup file 788 1650 98 172 > 173/429,pages2k_2156,FE23_northamerica_canada_cana169w,4.264518653507087,0.9602961986074566 ==================================================================== === POTENTIAL DUPLICATE 172/429: pages2k_2156+FE23_northamerica_canada_cana169w === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/NAm-Bonif.Schweingruber.1996-1.txt === === URL 2: https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/northamerica/canada/cana169w-noaa.rwl === True if pot_dup_corrs[i_pot_dups]>=0.98 else False False (len(time_1)==len(time_2)) False metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: False Automated choice. Metadata identical, automatically choose FE23 (Breitenmoser et al. (2014)) over PAGES 2k v2.2.0. conservative replication requirement KEEP RED CROSSES: remove pages2k_2156, keep FE23_northamerica_canada_cana169w. write decision to backup file 808 4692 98 173 > 174/429,pages2k_2214,iso2k_1631,0.0,1.0 ==================================================================== === POTENTIAL DUPLICATE 173/429: pages2k_2214+iso2k_1631 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Asi-Dasuopu.Thompson.2000.txt === === URL 2: https://www.ncdc.noaa.gov/paleo-search/study/11180 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: False data_identical: True correlation_perfect: True RECORDS IDENTICAL (identical data) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove pages2k_2214, keep iso2k_1631. write decision to backup file 813 2666 98 174 > 175/429,pages2k_2220,FE23_asia_russ127w,0.23755598128809655,0.9883046822070582 ==================================================================== === POTENTIAL DUPLICATE 174/429: pages2k_2220+FE23_asia_russ127w === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Asi-UKHEWW.Schweingruber.2002.txt === === URL 2: https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/asia/russ127w-noaa.rwl === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) False metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: False Automated choice. Metadata identical, automatically choose FE23 (Breitenmoser et al. (2014)) over PAGES 2k v2.2.0. conservative replication requirement KEEP RED CROSSES: remove pages2k_2220, keep FE23_asia_russ127w. write decision to backup file 816 2416 98 175 > 176/429,pages2k_2226,FE23_asia_mong007w,0.0,0.9988174986654474 ==================================================================== === POTENTIAL DUPLICATE 175/429: pages2k_2226+FE23_asia_mong007w === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Asi-MONG007.Schweingruber.2013.txt === === URL 2: https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/asia/mong007w-noaa.rwl === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) False metadata_identical: False lat True lon True elevation False archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: False
**Decision required for this duplicate pair (see figure above).**
saved figure in /home/jupyter-lluecke/dod2k_v2.0/dod2k/figs//dup_detection/all_merged/175_pages2k_2226_FE23_asia_mong007w__816_2416.jpg KEEP BLUE CIRCLES: keep pages2k_2226, remove FE23_asia_mong007w. write decision to backup file 833 2833 98 176 > 177/429,pages2k_2265,FE23_northamerica_usa_ak070,2.0092840563554346,0.9637690332839383 ==================================================================== === POTENTIAL DUPLICATE 176/429: pages2k_2265+FE23_northamerica_usa_ak070 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/NAm-WindyRidgeAlaska.DArrigo.2005.txt === === URL 2: https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/northamerica/usa/ak070-noaa.rwl === True if pot_dup_corrs[i_pot_dups]>=0.98 else False False (len(time_1)==len(time_2)) False metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: False Automated choice. Metadata identical, automatically choose FE23 (Breitenmoser et al. (2014)) over PAGES 2k v2.2.0. conservative replication requirement KEEP RED CROSSES: remove pages2k_2265, keep FE23_northamerica_usa_ak070. write decision to backup file 842 843 98 177 > 178/429,pages2k_2287,pages2k_2290,0.0,0.9999995540362934 ==================================================================== === POTENTIAL DUPLICATE 177/429: pages2k_2287+pages2k_2290 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-WesternAntarcticPeninsula.Shevenell.2011-1.txt === === URL 2: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-WesternAntarcticPeninsula.Shevenell.2011-1.txt === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: True data_identical: False correlation_perfect: True RECORDS IDENTICAL (perfect correlation). Automatically choose #1. KEEP BLUE CIRCLES: keep pages2k_2287, remove pages2k_2290. write decision to backup file 848 4183 98 178 > 179/429,pages2k_2300,ch2k_OS14RIP01_174,0.0,0.9999999872284353 ==================================================================== === POTENTIAL DUPLICATE 178/429: pages2k_2300+ch2k_OS14RIP01_174 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-PalauRockIslands.Osborne.2014.txt === === URL 2: https://www.ncdc.noaa.gov/paleo/study/16339 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: False lat True lon True elevation False archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: True RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove pages2k_2300, keep ch2k_OS14RIP01_174. write decision to backup file 850 2415 98 179 > 180/429,pages2k_2303,FE23_asia_mong006,0.37072945852236205,0.9987646516935693 ==================================================================== === POTENTIAL DUPLICATE 179/429: pages2k_2303+FE23_asia_mong006 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Asi-MONG006.Jacoby.2006.txt === === URL 2: https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/asia/mong006-noaa.rwl === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) False metadata_identical: False lat True lon True elevation False archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: False
**Decision required for this duplicate pair (see figure above).**
saved figure in /home/jupyter-lluecke/dod2k_v2.0/dod2k/figs//dup_detection/all_merged/179_pages2k_2303_FE23_asia_mong006__850_2415.jpg KEEP BLUE CIRCLES: keep pages2k_2303, remove FE23_asia_mong006. write decision to backup file 853 4197 98 180 > 181/429,pages2k_2309,ch2k_WE09ARR01_208,0.5077625559620448,0.9999999999999999 ==================================================================== === POTENTIAL DUPLICATE 180/429: pages2k_2309+ch2k_WE09ARR01_208 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-GBR.Wei.2009.txt === === URL 2: https://www.ncdc.noaa.gov/paleo/study/10425 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: True correlation_perfect: True RECORDS IDENTICAL (identical data) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove pages2k_2309, keep ch2k_WE09ARR01_208. write decision to backup file 854 4198 98 181 > 182/429,pages2k_2311,ch2k_WE09ARR01_210,0.5077625559620448,1.0 ==================================================================== === POTENTIAL DUPLICATE 181/429: pages2k_2311+ch2k_WE09ARR01_210 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-GBR.Wei.2009.txt === === URL 2: https://www.ncdc.noaa.gov/paleo/study/10425 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: True correlation_perfect: True RECORDS IDENTICAL (identical data) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove pages2k_2311, keep ch2k_WE09ARR01_210. write decision to backup file 858 2877 98 182 > 183/429,pages2k_2319,FE23_northamerica_usa_ak6,3.840937938296045,0.9731445668448404 ==================================================================== === POTENTIAL DUPLICATE 182/429: pages2k_2319+FE23_northamerica_usa_ak6 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/NAm-StarrigawanOldSitka.Kaiser.1996.txt === === URL 2: https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/northamerica/usa/ak6-noaa.rwl === True if pot_dup_corrs[i_pot_dups]>=0.98 else False False (len(time_1)==len(time_2)) False metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: False Automated choice. Metadata identical, automatically choose FE23 (Breitenmoser et al. (2014)) over PAGES 2k v2.2.0. conservative replication requirement KEEP RED CROSSES: remove pages2k_2319, keep FE23_northamerica_usa_ak6. write decision to backup file 862 864 98 183 > 184/429,pages2k_2339,pages2k_2344,0.0,0.9819400601513617 ==================================================================== === POTENTIAL DUPLICATE 183/429: pages2k_2339+pages2k_2344 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-TagusMudPatch.Abrantes.2005-1.txt === === URL 2: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-TagusMudPatch.Abrantes.2005-2.txt === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) False metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: False data_identical: False correlation_perfect: False
**Decision required for this duplicate pair (see figure above).**
saved figure in /home/jupyter-lluecke/dod2k_v2.0/dod2k/figs//dup_detection/all_merged/183_pages2k_2339_pages2k_2344__862_864.jpg KEEP RED CROSSES: remove pages2k_2339, keep pages2k_2344. write decision to backup file 870 4046 98 184 > 185/429,pages2k_2361,FE23_northamerica_usa_wa097,3.727634529190587,0.9933066574854877 ==================================================================== === POTENTIAL DUPLICATE 184/429: pages2k_2361+FE23_northamerica_usa_wa097 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/NAm-HohLakeHigh.Peterson.1994.txt === === URL 2: https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/northamerica/usa/wa097-noaa.rwl === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) False metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: False Automated choice. Metadata identical, automatically choose FE23 (Breitenmoser et al. (2014)) over PAGES 2k v2.2.0. conservative replication requirement KEEP RED CROSSES: remove pages2k_2361, keep FE23_northamerica_usa_wa097. write decision to backup file 883 3306 98 185 > 186/429,pages2k_2402,FE23_northamerica_usa_co586,4.672265454609224,0.9949546357141109 ==================================================================== === POTENTIAL DUPLICATE 185/429: pages2k_2402+FE23_northamerica_usa_co586 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/NAm-FoolCreek.Brown.2005-1.txt === === URL 2: https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/northamerica/usa/co586-noaa.rwl === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) False metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: False Automated choice. Metadata identical, automatically choose FE23 (Breitenmoser et al. (2014)) over PAGES 2k v2.2.0. conservative replication requirement KEEP RED CROSSES: remove pages2k_2402, keep FE23_northamerica_usa_co586. write decision to backup file 892 1610 98 186 > 187/429,pages2k_2430,FE23_northamerica_canada_cana113,6.064758659430112,0.9587868003899693 ==================================================================== === POTENTIAL DUPLICATE 186/429: pages2k_2430+FE23_northamerica_canada_cana113 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/NAm-ArrowsmithMountain.Briffa.1996-1.txt === === URL 2: https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/northamerica/canada/cana113-noaa.rwl === True if pot_dup_corrs[i_pot_dups]>=0.98 else False False (len(time_1)==len(time_2)) False metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: False Automated choice. Metadata identical, automatically choose FE23 (Breitenmoser et al. (2014)) over PAGES 2k v2.2.0. conservative replication requirement KEEP RED CROSSES: remove pages2k_2430, keep FE23_northamerica_canada_cana113. write decision to backup file 906 4103 98 187 > 188/429,pages2k_2473,FE23_northamerica_usa_wy022,0.0,0.9992211339017575 ==================================================================== === POTENTIAL DUPLICATE 187/429: pages2k_2473+FE23_northamerica_usa_wy022 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/NAm-MedicineBowPeak.Briffa.1996-1.txt === === URL 2: https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/northamerica/usa/wy022-noaa.rwl === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) False metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: False Automated choice. Metadata identical, automatically choose FE23 (Breitenmoser et al. (2014)) over PAGES 2k v2.2.0. conservative replication requirement KEEP RED CROSSES: remove pages2k_2473, keep FE23_northamerica_usa_wy022. write decision to backup file 915 916 98 188 > 189/429,pages2k_2500,pages2k_2502,0.0,1.0 ==================================================================== === POTENTIAL DUPLICATE 188/429: pages2k_2500+pages2k_2502 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-OkinawaTrough.Wu.2012-1.txt === === URL 2: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-OkinawaTrough.Wu.2012-2.txt === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) False metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: False data_identical: False correlation_perfect: False
**Decision required for this duplicate pair (see figure above).**
saved figure in /home/jupyter-lluecke/dod2k_v2.0/dod2k/figs//dup_detection/all_merged/188_pages2k_2500_pages2k_2502__915_916.jpg KEEP RED CROSSES: remove pages2k_2500, keep pages2k_2502. write decision to backup file 920 4690 98 189 > 190/429,pages2k_2510,iso2k_1626,0.0,1.0 ==================================================================== === POTENTIAL DUPLICATE 189/429: pages2k_2510+iso2k_1626 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Arc-Renland.Vinther.2008.txt === === URL 2: https://www.ncdc.noaa.gov/paleo/study/11148 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: False data_identical: True correlation_perfect: True RECORDS IDENTICAL (identical data) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove pages2k_2510, keep iso2k_1626. write decision to backup file 922 4652 98 190 > 191/429,pages2k_2514,iso2k_1467,0.0,1.0 ==================================================================== === POTENTIAL DUPLICATE 190/429: pages2k_2514+iso2k_1467 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-BuccooReefTobagoMontastrea.Moses.2006.txt === === URL 2: https://www.ncdc.noaa.gov/paleo/study/1924 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: False data_identical: True correlation_perfect: True RECORDS IDENTICAL (identical data) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove pages2k_2514, keep iso2k_1467. write decision to backup file 924 4593 98 191 > 192/429,pages2k_2517,iso2k_1130,0.0,0.9999999999999999 ==================================================================== === POTENTIAL DUPLICATE 191/429: pages2k_2517+iso2k_1130 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-Reunion.Pfeiffer.2004.txt === === URL 2: https://www.ncdc.noaa.gov/paleo/study/1890 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: True correlation_perfect: True RECORDS IDENTICAL (identical data) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove pages2k_2517, keep iso2k_1130. write decision to backup file 930 4681 98 192 > 193/429,pages2k_2534,iso2k_1575,0.13150147578139737,0.9999999992322688 ==================================================================== === POTENTIAL DUPLICATE 192/429: pages2k_2534+iso2k_1575 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ant-DSS.Moy.2012.txt === === URL 2: https://www.ncdc.noaa.gov/paleo-search/study/22589 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: True RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove pages2k_2534, keep iso2k_1575. write decision to backup file 932 4754 98 193 > 194/429,pages2k_2538,iso2k_1862,0.5755880921999212,0.999999999661364 ==================================================================== === POTENTIAL DUPLICATE 193/429: pages2k_2538+iso2k_1862 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ant-SiteDML05.Graf.2002.txt === === URL 2: https://doi.pangaea.de/10.1594/PANGAEA.104862 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: False lat True lon True elevation False archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: True RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove pages2k_2538, keep iso2k_1862. write decision to backup file 940 1590 98 194 > 195/429,pages2k_2561,FE23_northamerica_canada_cana094,4.317456281074108,0.9860766198395492 ==================================================================== === POTENTIAL DUPLICATE 194/429: pages2k_2561+FE23_northamerica_canada_cana094 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/NAm-BellMountain.Schweingruber.1996-1.txt === === URL 2: https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/northamerica/canada/cana094-noaa.rwl === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) False metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: False Automated choice. Metadata identical, automatically choose FE23 (Breitenmoser et al. (2014)) over PAGES 2k v2.2.0. conservative replication requirement KEEP RED CROSSES: remove pages2k_2561, keep FE23_northamerica_canada_cana094. write decision to backup file 948 950 98 195 > 196/429,pages2k_2592,pages2k_2596,0.0,1.0 ==================================================================== === POTENTIAL DUPLICATE 195/429: pages2k_2592+pages2k_2596 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-SouthAtlanticWestAfrica.Leduc.2010-1.txt === === URL 2: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-SouthAtlanticWestAfrica.Leduc.2010-2.txt === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) False metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: False data_identical: False correlation_perfect: False
**Decision required for this duplicate pair (see figure above).**
saved figure in /home/jupyter-lluecke/dod2k_v2.0/dod2k/figs//dup_detection/all_merged/195_pages2k_2592_pages2k_2596__948_950.jpg KEEP RED CROSSES: remove pages2k_2592, keep pages2k_2596. write decision to backup file 949 951 98 196 > 197/429,pages2k_2595,pages2k_2599,0.0,1.0 ==================================================================== === POTENTIAL DUPLICATE 196/429: pages2k_2595+pages2k_2599 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-SouthAtlanticWestAfrica.Leduc.2010-1.txt === === URL 2: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-SouthAtlanticWestAfrica.Leduc.2010-2.txt === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) False metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: False data_identical: False correlation_perfect: False
**Decision required for this duplicate pair (see figure above).**
saved figure in /home/jupyter-lluecke/dod2k_v2.0/dod2k/figs//dup_detection/all_merged/196_pages2k_2595_pages2k_2599__949_951.jpg KEEP RED CROSSES: remove pages2k_2595, keep pages2k_2599. write decision to backup file 954 955 98 197 > 198/429,pages2k_2604,pages2k_2606,0.0,0.9999999999998915 ==================================================================== === POTENTIAL DUPLICATE 197/429: pages2k_2604+pages2k_2606 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Arc-Lomonosovfonna.Divine.2011.txt === === URL 2: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Arc-Lomonosovfonna.Divine.2011.txt === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: True data_identical: False correlation_perfect: True RECORDS IDENTICAL (perfect correlation). Automatically choose #1. KEEP BLUE CIRCLES: keep pages2k_2604, remove pages2k_2606. write decision to backup file 954 4657 98 198 > 199/429,pages2k_2604,iso2k_1481,0.5335020039767102,0.9999999999774063 ==================================================================== === POTENTIAL DUPLICATE 198/429: pages2k_2604+iso2k_1481 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Arc-Lomonosovfonna.Divine.2011.txt === === URL 2: https://www.ncdc.noaa.gov/paleo/study/22541 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: True RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove pages2k_2604, keep iso2k_1481. write decision to backup file 955 4657 98 199 > 200/429,pages2k_2606,iso2k_1481,0.5335020039767102,0.999999999977323 ==================================================================== === POTENTIAL DUPLICATE 199/429: pages2k_2606+iso2k_1481 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Arc-Lomonosovfonna.Divine.2011.txt === === URL 2: https://www.ncdc.noaa.gov/paleo/study/22541 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: True RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove pages2k_2606, keep iso2k_1481. write decision to backup file 956 957 98 200 > 201/429,pages2k_2607,pages2k_2609,0.0,0.9993063378972388 ==================================================================== === POTENTIAL DUPLICATE 200/429: pages2k_2607+pages2k_2609 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-DryTortugas.Lund.2006-1.txt === === URL 2: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-DryTortugas.Lund.2006-1.txt === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: True data_identical: False correlation_perfect: True RECORDS IDENTICAL (perfect correlation). Automatically choose #1. KEEP BLUE CIRCLES: keep pages2k_2607, remove pages2k_2609. write decision to backup file 956 959 98 201 > 202/429,pages2k_2607,pages2k_2612,0.0,0.9993063378972388 ==================================================================== === POTENTIAL DUPLICATE 201/429: pages2k_2607+pages2k_2612 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-DryTortugas.Lund.2006-1.txt === === URL 2: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-DryTortugas.Lund.2006-2.txt === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: False data_identical: False correlation_perfect: True RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #1. KEEP BLUE CIRCLES: keep pages2k_2607, remove pages2k_2612. write decision to backup file 957 959 98 202 > 203/429,pages2k_2609,pages2k_2612,0.0,1.0 ==================================================================== === POTENTIAL DUPLICATE 202/429: pages2k_2609+pages2k_2612 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-DryTortugas.Lund.2006-1.txt === === URL 2: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-DryTortugas.Lund.2006-2.txt === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: False data_identical: True correlation_perfect: True RECORDS IDENTICAL (identical data) except for metadata. Automatically choose #1. KEEP BLUE CIRCLES: keep pages2k_2609, remove pages2k_2612. write decision to backup file 960 4653 98 203 > 204/429,pages2k_2613,iso2k_1470,0.0,1.0 ==================================================================== === POTENTIAL DUPLICATE 203/429: pages2k_2613+iso2k_1470 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-BuccooReefTobagoSidereastrea.Moses.2006-1.txt === === URL 2: https://www.ncdc.noaa.gov/paleo/study/1924 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: False data_identical: True correlation_perfect: True RECORDS IDENTICAL (identical data) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove pages2k_2613, keep iso2k_1470. write decision to backup file 962 4680 98 204 > 205/429,pages2k_2617,iso2k_1573,0.0,1.0 ==================================================================== === POTENTIAL DUPLICATE 204/429: pages2k_2617+iso2k_1573 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-ClippertonAtoll.Linsley.2000.txt === === URL 2: https://www.ncdc.noaa.gov/paleo/study/1846 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: False data_identical: True correlation_perfect: True RECORDS IDENTICAL (identical data) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove pages2k_2617, keep iso2k_1573. write decision to backup file 970 3402 98 205 > 206/429,pages2k_2634,FE23_northamerica_usa_id013,1.8532231170311824,0.9748413733893866 ==================================================================== === POTENTIAL DUPLICATE 205/429: pages2k_2634+FE23_northamerica_usa_id013 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/NAm-SleepingDeerRoad.Hughes.2005-1.txt === === URL 2: https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/northamerica/usa/id013-noaa.rwl === True if pot_dup_corrs[i_pot_dups]>=0.98 else False False (len(time_1)==len(time_2)) False metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: False Automated choice. Metadata identical, automatically choose FE23 (Breitenmoser et al. (2014)) over PAGES 2k v2.2.0. conservative replication requirement KEEP RED CROSSES: remove pages2k_2634, keep FE23_northamerica_usa_id013. write decision to backup file 978 2777 98 206 > 207/429,pages2k_2660,FE23_northamerica_usa_ak014,5.559669351093426,0.9947190610483321 ==================================================================== === POTENTIAL DUPLICATE 206/429: pages2k_2660+FE23_northamerica_usa_ak014 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/NAm-SettlementPointAfognakIsland.Harlan.1998.txt === === URL 2: https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/northamerica/usa/ak014-noaa.rwl === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) False metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: False Automated choice. Metadata identical, automatically choose FE23 (Breitenmoser et al. (2014)) over PAGES 2k v2.2.0. conservative replication requirement KEEP RED CROSSES: remove pages2k_2660, keep FE23_northamerica_usa_ak014. write decision to backup file 984 4104 98 207 > 208/429,pages2k_2677,FE23_northamerica_usa_wy023,4.556051501980059,0.9789527632198499 ==================================================================== === POTENTIAL DUPLICATE 207/429: pages2k_2677+FE23_northamerica_usa_wy023 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/NAm-SylvanPassbeiCody.Briffa.1996-1.txt === === URL 2: https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/northamerica/usa/wy023-noaa.rwl === True if pot_dup_corrs[i_pot_dups]>=0.98 else False False (len(time_1)==len(time_2)) False metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: False Automated choice. Metadata identical, automatically choose FE23 (Breitenmoser et al. (2014)) over PAGES 2k v2.2.0. conservative replication requirement KEEP RED CROSSES: remove pages2k_2677, keep FE23_northamerica_usa_wy023. write decision to backup file 992 2856 98 208 > 209/429,pages2k_2703,FE23_northamerica_usa_ak094,5.851894956811684,0.9838935021941124 ==================================================================== === POTENTIAL DUPLICATE 208/429: pages2k_2703+FE23_northamerica_usa_ak094 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/NAm-PrinceWilliamSound.Barclay.1999.txt === === URL 2: https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/northamerica/usa/ak094-noaa.rwl === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) False metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: False Automated choice. Metadata identical, automatically choose FE23 (Breitenmoser et al. (2014)) over PAGES 2k v2.2.0. conservative replication requirement KEEP RED CROSSES: remove pages2k_2703, keep FE23_northamerica_usa_ak094. write decision to backup file 999 1719 98 209 > 210/429,pages2k_2722,FE23_northamerica_canada_cana238,3.5216567655985758,0.9997808243683045 ==================================================================== === POTENTIAL DUPLICATE 209/429: pages2k_2722+FE23_northamerica_canada_cana238 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/NAm-CornwallHills.Wilson.2005-1.txt === === URL 2: https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/northamerica/canada/cana238-noaa.rwl === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) False metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: False Automated choice. Metadata identical, automatically choose FE23 (Breitenmoser et al. (2014)) over PAGES 2k v2.2.0. conservative replication requirement KEEP RED CROSSES: remove pages2k_2722, keep FE23_northamerica_canada_cana238. write decision to backup file 1009 4707 98 210 > 211/429,pages2k_2750,iso2k_1708,0.3070428433390891,0.9999999999987222 ==================================================================== === POTENTIAL DUPLICATE 210/429: pages2k_2750+iso2k_1708 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ant-JamesRossIsland.Mulvaney.2013.txt === === URL 2: https://www.ncdc.noaa.gov/paleo/study/13954 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: False data_identical: False correlation_perfect: True RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove pages2k_2750, keep iso2k_1708. write decision to backup file 1010 1011 98 211 > 212/429,pages2k_2752,pages2k_2755,0.0,0.9994757834201398 ==================================================================== === POTENTIAL DUPLICATE 211/429: pages2k_2752+pages2k_2755 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-GreatBahamaBank.Lund.2006-1.txt === === URL 2: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-GreatBahamaBank.Lund.2006-1.txt === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: True data_identical: False correlation_perfect: True RECORDS IDENTICAL (perfect correlation). Automatically choose #1. KEEP BLUE CIRCLES: keep pages2k_2752, remove pages2k_2755. write decision to backup file 1010 1013 98 212 > 213/429,pages2k_2752,pages2k_2759,0.0,0.9994757834201398 ==================================================================== === POTENTIAL DUPLICATE 212/429: pages2k_2752+pages2k_2759 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-GreatBahamaBank.Lund.2006-1.txt === === URL 2: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-GreatBahamaBank.Lund.2006-2.txt === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: False data_identical: False correlation_perfect: True RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #1. KEEP BLUE CIRCLES: keep pages2k_2752, remove pages2k_2759. write decision to backup file 1011 1013 98 213 > 214/429,pages2k_2755,pages2k_2759,0.0,1.0 ==================================================================== === POTENTIAL DUPLICATE 213/429: pages2k_2755+pages2k_2759 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-GreatBahamaBank.Lund.2006-1.txt === === URL 2: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-GreatBahamaBank.Lund.2006-2.txt === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: False data_identical: True correlation_perfect: True RECORDS IDENTICAL (identical data) except for metadata. Automatically choose #1. KEEP BLUE CIRCLES: keep pages2k_2755, remove pages2k_2759. write decision to backup file 1029 1030 98 214 > 215/429,pages2k_2793,pages2k_2795,0.0,0.9980751700561887 ==================================================================== === POTENTIAL DUPLICATE 214/429: pages2k_2793+pages2k_2795 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-EasternTropicalNorthAtlantic.Kuhnert.2011-1.txt === === URL 2: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-EasternTropicalNorthAtlantic.Kuhnert.2011-1.txt === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: True data_identical: False correlation_perfect: True RECORDS IDENTICAL (perfect correlation). Automatically choose #1. KEEP BLUE CIRCLES: keep pages2k_2793, remove pages2k_2795. write decision to backup file 1030 1032 98 215 > 216/429,pages2k_2795,pages2k_2798,0.0,0.9808797757205986 ==================================================================== === POTENTIAL DUPLICATE 215/429: pages2k_2795+pages2k_2798 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-EasternTropicalNorthAtlantic.Kuhnert.2011-1.txt === === URL 2: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-EasternTropicalNorthAtlantic.Kuhnert.2011-2.txt === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) False metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: False data_identical: False correlation_perfect: False
**Decision required for this duplicate pair (see figure above).** --------------------------------------------------------------------------------------------------------- ***ATTENTION*** THIS RECORD IS ASSOCIATED WITH MULTIPLE DUPLICATES! PLEASE PAY SPECIAL ATTENTION WHEN MAKING DECISIONS FOR THIS RECORD! The potential duplicates also associated with this record are: - pages2k_2793 ---------------------------------------------------------------------------------------------------------
saved figure in /home/jupyter-lluecke/dod2k_v2.0/dod2k/figs//dup_detection/all_merged/215_pages2k_2795_pages2k_2798__1030_1032.jpg KEEP BOTH: keep pages2k_2795, keep pages2k_2798. write decision to backup file 1031 1032 98 216 > 217/429,pages2k_2796,pages2k_2798,0.0,0.9982162224217608 ==================================================================== === POTENTIAL DUPLICATE 216/429: pages2k_2796+pages2k_2798 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-EasternTropicalNorthAtlantic.Kuhnert.2011-2.txt === === URL 2: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-EasternTropicalNorthAtlantic.Kuhnert.2011-2.txt === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: True data_identical: False correlation_perfect: True RECORDS IDENTICAL (perfect correlation). Automatically choose #1. KEEP BLUE CIRCLES: keep pages2k_2796, remove pages2k_2798. write decision to backup file 1043 2386 98 217 > 218/429,pages2k_2830,FE23_northamerica_mexico_mexi020,3.706446234062073,0.9991748202674907 ==================================================================== === POTENTIAL DUPLICATE 217/429: pages2k_2830+FE23_northamerica_mexico_mexi020 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/NAm-LaTasajera.Biondi.2001.txt === === URL 2: https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/northamerica/mexico/mexi020-noaa.rwl === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) False metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: False Automated choice. Metadata identical, automatically choose FE23 (Breitenmoser et al. (2014)) over PAGES 2k v2.2.0. conservative replication requirement KEEP RED CROSSES: remove pages2k_2830, keep FE23_northamerica_mexico_mexi020. write decision to backup file 1047 4032 98 218 > 219/429,pages2k_2843,FE23_northamerica_usa_wa083,2.4462606401401104,0.9826298673983576 ==================================================================== === POTENTIAL DUPLICATE 218/429: pages2k_2843+FE23_northamerica_usa_wa083 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/NAm-ShermanCreekPass.Briffa.1996-1.txt === === URL 2: https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/northamerica/usa/wa083-noaa.rwl === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) False metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: False Automated choice. Metadata identical, automatically choose FE23 (Breitenmoser et al. (2014)) over PAGES 2k v2.2.0. conservative replication requirement KEEP RED CROSSES: remove pages2k_2843, keep FE23_northamerica_usa_wa083. write decision to backup file 1066 1067 98 219 > 220/429,pages2k_2899,pages2k_2901,0.0,0.9999406235904933 ==================================================================== === POTENTIAL DUPLICATE 219/429: pages2k_2899+pages2k_2901 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-KuroshioCurrent.Isono.2009-1.txt === === URL 2: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-KuroshioCurrent.Isono.2009-2.txt === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) False metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: False data_identical: False correlation_perfect: False
**Decision required for this duplicate pair (see figure above).**
saved figure in /home/jupyter-lluecke/dod2k_v2.0/dod2k/figs//dup_detection/all_merged/219_pages2k_2899_pages2k_2901__1066_1067.jpg KEEP RED CROSSES: remove pages2k_2899, keep pages2k_2901. write decision to backup file 1068 1069 98 220 > 221/429,pages2k_2904,pages2k_2906,0.0,0.9975794070998795 ==================================================================== === POTENTIAL DUPLICATE 220/429: pages2k_2904+pages2k_2906 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-NorthwestPacificOcean.Harada.2004-1.txt === === URL 2: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-NorthwestPacificOcean.Harada.2004-2.txt === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) False metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: False data_identical: False correlation_perfect: False
**Decision required for this duplicate pair (see figure above).**
saved figure in /home/jupyter-lluecke/dod2k_v2.0/dod2k/figs//dup_detection/all_merged/220_pages2k_2904_pages2k_2906__1068_1069.jpg KEEP BLUE CIRCLES: keep pages2k_2904, remove pages2k_2906. write decision to backup file 1075 3151 98 221 > 222/429,pages2k_2922,FE23_northamerica_usa_ca603,3.4621315803107966,0.957962036670777 ==================================================================== === POTENTIAL DUPLICATE 221/429: pages2k_2922+FE23_northamerica_usa_ca603 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/NAm-DanaPlateauInyoNationalForest.King.2000.txt === === URL 2: https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/northamerica/usa/ca603-noaa.rwl === True if pot_dup_corrs[i_pot_dups]>=0.98 else False False (len(time_1)==len(time_2)) False metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: False Automated choice. Metadata identical, automatically choose FE23 (Breitenmoser et al. (2014)) over PAGES 2k v2.2.0. conservative replication requirement KEEP RED CROSSES: remove pages2k_2922, keep FE23_northamerica_usa_ca603. write decision to backup file 1086 4480 98 222 > 223/429,pages2k_2953,iso2k_573,0.0,0.9999999999999999 ==================================================================== === POTENTIAL DUPLICATE 222/429: pages2k_2953+iso2k_573 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ant-DomeC.Stenni.2001.txt === === URL 2: https://www.ncdc.noaa.gov/paleo-search/study/2492 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: True RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove pages2k_2953, keep iso2k_573. write decision to backup file 1088 2406 98 223 > 224/429,pages2k_2959,FE23_northamerica_mexico_mexi043,3.7064462340624265,0.9950845780658646 ==================================================================== === POTENTIAL DUPLICATE 223/429: pages2k_2959+FE23_northamerica_mexico_mexi043 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/NAm-CienegadeNuestraSenoradeGuadalupe.Fule.2009.txt === === URL 2: https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/northamerica/mexico/mexi043-noaa.rwl === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) False metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: False Automated choice. Metadata identical, automatically choose FE23 (Breitenmoser et al. (2014)) over PAGES 2k v2.2.0. conservative replication requirement KEEP RED CROSSES: remove pages2k_2959, keep FE23_northamerica_mexico_mexi043. write decision to backup file 1094 3397 98 224 > 225/429,pages2k_2976,FE23_northamerica_usa_id008,3.9402844249478353,0.9998526831248677 ==================================================================== === POTENTIAL DUPLICATE 224/429: pages2k_2976+FE23_northamerica_usa_id008 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/NAm-GalenaPassSawtooth.Briffa.1996-1.txt === === URL 2: https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/northamerica/usa/id008-noaa.rwl === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) False metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: False Automated choice. Metadata identical, automatically choose FE23 (Breitenmoser et al. (2014)) over PAGES 2k v2.2.0. conservative replication requirement KEEP RED CROSSES: remove pages2k_2976, keep FE23_northamerica_usa_id008. write decision to backup file 1102 3768 98 225 > 226/429,pages2k_3002,FE23_northamerica_usa_or043,4.326745645368244,0.991561064546057 ==================================================================== === POTENTIAL DUPLICATE 225/429: pages2k_3002+FE23_northamerica_usa_or043 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/NAm-BarlowPassamMtHood.Briffa.1996-1.txt === === URL 2: https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/northamerica/usa/or043-noaa.rwl === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) False metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: False Automated choice. Metadata identical, automatically choose FE23 (Breitenmoser et al. (2014)) over PAGES 2k v2.2.0. conservative replication requirement KEEP RED CROSSES: remove pages2k_3002, keep FE23_northamerica_usa_or043. write decision to backup file 1111 1112 98 226 > 227/429,pages2k_3028,pages2k_3030,0.0,0.9996177606042984 ==================================================================== === POTENTIAL DUPLICATE 226/429: pages2k_3028+pages2k_3030 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-GreatBahamaBank.Richter.2006-1.txt === === URL 2: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-GreatBahamaBank.Richter.2006-1.txt === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: True data_identical: False correlation_perfect: True RECORDS IDENTICAL (perfect correlation). Automatically choose #1. KEEP BLUE CIRCLES: keep pages2k_3028, remove pages2k_3030. write decision to backup file 1111 1114 98 227 > 228/429,pages2k_3028,pages2k_3033,0.0,0.9996177606042984 ==================================================================== === POTENTIAL DUPLICATE 227/429: pages2k_3028+pages2k_3033 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-GreatBahamaBank.Richter.2006-1.txt === === URL 2: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-GreatBahamaBank.Richter.2006-2.txt === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: False data_identical: False correlation_perfect: True RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #1. KEEP BLUE CIRCLES: keep pages2k_3028, remove pages2k_3033. write decision to backup file 1112 1114 98 228 > 229/429,pages2k_3030,pages2k_3033,0.0,1.0 ==================================================================== === POTENTIAL DUPLICATE 228/429: pages2k_3030+pages2k_3033 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-GreatBahamaBank.Richter.2006-1.txt === === URL 2: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-GreatBahamaBank.Richter.2006-2.txt === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: False data_identical: True correlation_perfect: True RECORDS IDENTICAL (identical data) except for metadata. Automatically choose #1. KEEP BLUE CIRCLES: keep pages2k_3030, remove pages2k_3033. write decision to backup file 1116 3543 98 229 > 230/429,pages2k_3038,FE23_northamerica_usa_mt108,6.131312678586471,0.9782486241361592 ==================================================================== === POTENTIAL DUPLICATE 229/429: pages2k_3038+FE23_northamerica_usa_mt108 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/NAm-HighlandFireOutlook.Briffa.1996-1.txt === === URL 2: https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/northamerica/usa/mt108-noaa.rwl === True if pot_dup_corrs[i_pot_dups]>=0.98 else False False (len(time_1)==len(time_2)) False metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: False Automated choice. Metadata identical, automatically choose FE23 (Breitenmoser et al. (2014)) over PAGES 2k v2.2.0. conservative replication requirement KEEP RED CROSSES: remove pages2k_3038, keep FE23_northamerica_usa_mt108. write decision to backup file 1125 4500 98 230 > 231/429,pages2k_3064,iso2k_698,0.0,1.0 ==================================================================== === POTENTIAL DUPLICATE 230/429: pages2k_3064+iso2k_698 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-NorthEastBreakersBermuda.Kuhnert.2002-1.txt === === URL 2: https://www.ncdc.noaa.gov/paleo/study/1872 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: False data_identical: True correlation_perfect: True RECORDS IDENTICAL (identical data) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove pages2k_3064, keep iso2k_698. write decision to backup file 1126 4316 98 231 > 232/429,pages2k_3068,ch2k_ZI14IFR02_522,1.1668761630139568,1.0 ==================================================================== === POTENTIAL DUPLICATE 231/429: pages2k_3068+ch2k_ZI14IFR02_522 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-Ifaty1.Zinke.2014.txt === === URL 2: https://www.ncdc.noaa.gov/paleo/study/16438 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: True correlation_perfect: True RECORDS IDENTICAL (identical data) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove pages2k_3068, keep ch2k_ZI14IFR02_522. write decision to backup file 1126 4317 98 232 > 233/429,pages2k_3068,ch2k_ZI14IFR02_524,1.1668761630139568,1.0 ==================================================================== === POTENTIAL DUPLICATE 232/429: pages2k_3068+ch2k_ZI14IFR02_524 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-Ifaty1.Zinke.2014.txt === === URL 2: https://www.ncdc.noaa.gov/paleo/study/16438 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: True correlation_perfect: True RECORDS IDENTICAL (identical data) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove pages2k_3068, keep ch2k_ZI14IFR02_524. write decision to backup file 1132 4173 98 233 > 234/429,pages2k_3085,ch2k_KU00NIN01_150,0.0,1.0 ==================================================================== === POTENTIAL DUPLICATE 233/429: pages2k_3085+ch2k_KU00NIN01_150 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-Ningaloo.Kuhnert.2000.txt === === URL 2: https://www.ncdc.noaa.gov/paleo/study/1867 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: True RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove pages2k_3085, keep ch2k_KU00NIN01_150. write decision to backup file 1132 4672 98 234 > 235/429,pages2k_3085,iso2k_1554,0.0,1.0 ==================================================================== === POTENTIAL DUPLICATE 234/429: pages2k_3085+iso2k_1554 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-Ningaloo.Kuhnert.2000.txt === === URL 2: https://www.ncdc.noaa.gov/paleo/study/1867 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: True correlation_perfect: True RECORDS IDENTICAL (identical data) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove pages2k_3085, keep iso2k_1554. write decision to backup file 1132 4673 98 235 > 236/429,pages2k_3085,iso2k_1556,0.0,0.9995653924115724 ==================================================================== === POTENTIAL DUPLICATE 235/429: pages2k_3085+iso2k_1556 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-Ningaloo.Kuhnert.2000.txt === === URL 2: https://www.ncdc.noaa.gov/paleo/study/1867 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) False metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: False
**Decision required for this duplicate pair (see figure above).**
---------------------------------------------------------------------------------------------------------
***ATTENTION*** THIS RECORD IS ASSOCIATED WITH MULTIPLE DUPLICATES! PLEASE PAY SPECIAL ATTENTION WHEN MAKING DECISIONS FOR THIS RECORD!
The potential duplicates also associated with this record are:
............................................................
- Dataset ID : ch2k_KU00NIN01_150
- URL : https://www.ncdc.noaa.gov/paleo/study/1867
............................................................
- Dataset ID : iso2k_1554
- URL : https://www.ncdc.noaa.gov/paleo/study/1867
---------------------------------------------------------------------------------------------------------
saved figure in /home/jupyter-lluecke/dod2k_v2.0/dod2k/figs//dup_detection/all_merged/235_pages2k_3085_iso2k_1556__1132_4673.jpg KEEP RED CROSSES: remove pages2k_3085, keep iso2k_1556. write decision to backup file 1140 3274 98 236 > 237/429,pages2k_3107,FE23_northamerica_usa_co552,1.4626835452261282,0.9441889459422474 ==================================================================== === POTENTIAL DUPLICATE 236/429: pages2k_3107+FE23_northamerica_usa_co552 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/NAm-RedMountainPassSilverton.Graybill.1994-1.txt === === URL 2: https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/northamerica/usa/co552-noaa.rwl === True if pot_dup_corrs[i_pot_dups]>=0.98 else False False (len(time_1)==len(time_2)) False metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: False Automated choice. Metadata identical, automatically choose FE23 (Breitenmoser et al. (2014)) over PAGES 2k v2.2.0. conservative replication requirement KEEP RED CROSSES: remove pages2k_3107, keep FE23_northamerica_usa_co552. write decision to backup file 1141 3274 98 237 > 238/429,pages2k_3108,FE23_northamerica_usa_co552,1.4626835452261282,0.9121185067546479 ==================================================================== === POTENTIAL DUPLICATE 237/429: pages2k_3108+FE23_northamerica_usa_co552 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/NAm-RedMountainPassSilverton.Graybill.1994-1.txt === === URL 2: https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/northamerica/usa/co552-noaa.rwl === True if pot_dup_corrs[i_pot_dups]>=0.98 else False False (len(time_1)==len(time_2)) False metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: False Automated choice. Metadata identical, automatically choose FE23 (Breitenmoser et al. (2014)) over PAGES 2k v2.2.0. conservative replication requirement KEEP RED CROSSES: remove pages2k_3108, keep FE23_northamerica_usa_co552. write decision to backup file 1149 4170 98 238 > 239/429,pages2k_3132,ch2k_QU06RAB01_144,1.3265164823622553,0.9998939017468722 ==================================================================== === POTENTIAL DUPLICATE 238/429: pages2k_3132+ch2k_QU06RAB01_144 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-Rabaul.Quinn.2006.txt === === URL 2: https://www.ncdc.noaa.gov/paleo/study/6116 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: True RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove pages2k_3132, keep ch2k_QU06RAB01_144. write decision to backup file 1149 4626 98 239 > 240/429,pages2k_3132,iso2k_1311,0.0,0.9999999891930905 ==================================================================== === POTENTIAL DUPLICATE 239/429: pages2k_3132+iso2k_1311 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-Rabaul.Quinn.2006.txt === === URL 2: https://www.ncdc.noaa.gov/paleo/study/6116 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: False data_identical: False correlation_perfect: True RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove pages2k_3132, keep iso2k_1311. write decision to backup file 1150 4171 98 240 > 241/429,pages2k_3134,ch2k_QU06RAB01_146,1.3265164823622553,0.9999849183468115 ==================================================================== === POTENTIAL DUPLICATE 240/429: pages2k_3134+ch2k_QU06RAB01_146 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-Rabaul.Quinn.2006.txt === === URL 2: https://www.ncdc.noaa.gov/paleo/study/6116 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: True RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove pages2k_3134, keep ch2k_QU06RAB01_146. write decision to backup file 1164 2524 98 241 > 242/429,pages2k_3170,FE23_australia_newz062,0.4672268985752102,0.989769537189296 ==================================================================== === POTENTIAL DUPLICATE 241/429: pages2k_3170+FE23_australia_newz062 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Aus-TakapariCedar.Xiong.2000.txt === === URL 2: https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/australia/newz062-noaa.rwl === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) False metadata_identical: False lat True lon True elevation False archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: False Record 2 (FE23_australia_newz062) is UPDATE of record 1 (pages2k_3170). Automatically choose 2. KEEP RED CROSSES: remove pages2k_3170, keep FE23_australia_newz062. write decision to backup file 1167 2820 98 242 > 243/429,pages2k_3179,FE23_northamerica_usa_ak057,1.8527989414495474,0.9986759093327194 ==================================================================== === POTENTIAL DUPLICATE 242/429: pages2k_3179+FE23_northamerica_usa_ak057 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/NAm-AlmondButterLower.DArrigo.2005.txt === === URL 2: https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/northamerica/usa/ak057-noaa.rwl === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) False metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: False Automated choice. Metadata identical, automatically choose FE23 (Breitenmoser et al. (2014)) over PAGES 2k v2.2.0. conservative replication requirement KEEP RED CROSSES: remove pages2k_3179, keep FE23_northamerica_usa_ak057. write decision to backup file 1170 1171 98 243 > 244/429,pages2k_3188,pages2k_3191,0.0,0.999688231485937 ==================================================================== === POTENTIAL DUPLICATE 243/429: pages2k_3188+pages2k_3191 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-RAPiD-12-1K.Thornalley.2009-1.txt === === URL 2: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-RAPiD-12-1K.Thornalley.2009-1.txt === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: True data_identical: False correlation_perfect: True RECORDS IDENTICAL (perfect correlation). Automatically choose #1. KEEP BLUE CIRCLES: keep pages2k_3188, remove pages2k_3191. write decision to backup file 1172 2420 98 244 > 245/429,pages2k_3196,FE23_asia_mong011,2.4341761943752602,0.9999999797688458 ==================================================================== === POTENTIAL DUPLICATE 244/429: pages2k_3196+FE23_asia_mong011 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Asi-MONG011.Jacoby.2006.txt === === URL 2: https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/asia/mong011-noaa.rwl === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) False metadata_identical: False lat True lon True elevation False archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: False
**Decision required for this duplicate pair (see figure above).**
saved figure in /home/jupyter-lluecke/dod2k_v2.0/dod2k/figs//dup_detection/all_merged/244_pages2k_3196_FE23_asia_mong011__1172_2420.jpg KEEP BLUE CIRCLES: keep pages2k_3196, remove FE23_asia_mong011. write decision to backup file 1175 4712 98 245 > 246/429,pages2k_3202,iso2k_1727,0.0,0.9999999999999999 ==================================================================== === POTENTIAL DUPLICATE 245/429: pages2k_3202+iso2k_1727 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Arc-DevonIceCap.Fisher.1983.txt === === URL 2: https://www1.ncdc.noaa.gov/pub/data/paleo/icecore/polar/devon/d7273del_5yr.txt === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: False data_identical: True correlation_perfect: True RECORDS IDENTICAL (identical data) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove pages2k_3202, keep iso2k_1727. write decision to backup file 1185 1186 98 246 > 247/429,pages2k_3234,pages2k_3236,0.0,0.9992445070124597 ==================================================================== === POTENTIAL DUPLICATE 246/429: pages2k_3234+pages2k_3236 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-CariacoBasin.Black.2007-1.txt === === URL 2: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-CariacoBasin.Black.2007-1.txt === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: True data_identical: False correlation_perfect: True RECORDS IDENTICAL (perfect correlation). Automatically choose #1. KEEP BLUE CIRCLES: keep pages2k_3234, remove pages2k_3236. write decision to backup file 1185 1188 98 247 > 248/429,pages2k_3234,pages2k_3239,0.0,0.9992445070124597 ==================================================================== === POTENTIAL DUPLICATE 247/429: pages2k_3234+pages2k_3239 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-CariacoBasin.Black.2007-1.txt === === URL 2: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-CariacoBasin.Black.2007-2.txt === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: False data_identical: False correlation_perfect: True RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #1. KEEP BLUE CIRCLES: keep pages2k_3234, remove pages2k_3239. write decision to backup file 1186 1188 98 248 > 249/429,pages2k_3236,pages2k_3239,0.0,1.0 ==================================================================== === POTENTIAL DUPLICATE 248/429: pages2k_3236+pages2k_3239 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-CariacoBasin.Black.2007-1.txt === === URL 2: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-CariacoBasin.Black.2007-2.txt === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: False data_identical: True correlation_perfect: True RECORDS IDENTICAL (identical data) except for metadata. Automatically choose #1. KEEP BLUE CIRCLES: keep pages2k_3236, remove pages2k_3239. write decision to backup file 1191 4339 98 249 > 250/429,pages2k_3243,iso2k_0,2.634776316918305,0.9999999999999998 ==================================================================== === POTENTIAL DUPLICATE 249/429: pages2k_3243+iso2k_0 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-Clipperton1b.Wu.2014.txt === === URL 2: https://www.ncdc.noaa.gov/paleo/study/17380 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: True RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove pages2k_3243, keep iso2k_0. write decision to backup file 1198 4613 98 250 > 251/429,pages2k_3263,iso2k_1264,0.0,0.9999999999999999 ==================================================================== === POTENTIAL DUPLICATE 250/429: pages2k_3263+iso2k_1264 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-SavusavuBay1F.Linsley.2006.txt === === URL 2: https://www.ncdc.noaa.gov/paleo/study/16216 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: False data_identical: True correlation_perfect: True RECORDS IDENTICAL (identical data) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove pages2k_3263, keep iso2k_1264. write decision to backup file 1200 4269 98 251 > 252/429,pages2k_3266,ch2k_GO12SBV01_396,3.209089318097252,0.9999273362756578 ==================================================================== === POTENTIAL DUPLICATE 251/429: pages2k_3266+ch2k_GO12SBV01_396 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-Vanuatu.Gorman.2012.txt === === URL 2: https://www.ncdc.noaa.gov/paleo/study/13439 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: True RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove pages2k_3266, keep ch2k_GO12SBV01_396. write decision to backup file 1200 4538 98 252 > 253/429,pages2k_3266,iso2k_870,0.0,0.999999992571119 ==================================================================== === POTENTIAL DUPLICATE 252/429: pages2k_3266+iso2k_870 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-Vanuatu.Gorman.2012.txt === === URL 2: https://www.ncdc.noaa.gov/paleo/study/13439 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: False data_identical: False correlation_perfect: True RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove pages2k_3266, keep iso2k_870. write decision to backup file 1217 4417 98 253 > 254/429,pages2k_3307,iso2k_339,0.0,1.0 ==================================================================== === POTENTIAL DUPLICATE 253/429: pages2k_3307+iso2k_339 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Arc-CampCentury.Fisher.1969.txt === === URL 2: https://www.ncdc.noaa.gov/paleo/study/11148 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: False lat True lon True elevation False archivetype True paleodata_proxy True sites_identical: True URL_identical: False data_identical: True correlation_perfect: True RECORDS IDENTICAL (identical data) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove pages2k_3307, keep iso2k_339. write decision to backup file 1219 3109 98 254 > 255/429,pages2k_3313,FE23_northamerica_usa_ca560,4.3933357623831375,0.9971426205961806 ==================================================================== === POTENTIAL DUPLICATE 254/429: pages2k_3313+FE23_northamerica_usa_ca560 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/NAm-YosemiteParkEEingang.Briffa.1996-1.txt === === URL 2: https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/northamerica/usa/ca560-noaa.rwl === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) False metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: False Automated choice. Metadata identical, automatically choose FE23 (Breitenmoser et al. (2014)) over PAGES 2k v2.2.0. conservative replication requirement KEEP RED CROSSES: remove pages2k_3313, keep FE23_northamerica_usa_ca560. write decision to backup file 1227 1229 98 255 > 256/429,pages2k_3337,pages2k_3342,0.0,1.0 ==================================================================== === POTENTIAL DUPLICATE 255/429: pages2k_3337+pages2k_3342 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-CapeGhir.McGregor.2007-1.txt === === URL 2: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-CapeGhir.McGregor.2007-2.txt === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: False data_identical: False correlation_perfect: True RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #1. KEEP BLUE CIRCLES: keep pages2k_3337, remove pages2k_3342. write decision to backup file 1233 4301 98 256 > 257/429,pages2k_3352,ch2k_ZI14TUR01_480,0.0,1.0 ==================================================================== === POTENTIAL DUPLICATE 256/429: pages2k_3352+ch2k_ZI14TUR01_480 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-IfatyTul3.Zinke.2014.txt === === URL 2: https://www.ncdc.noaa.gov/paleo/study/16438 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: True correlation_perfect: True RECORDS IDENTICAL (identical data) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove pages2k_3352, keep ch2k_ZI14TUR01_480. write decision to backup file 1233 4302 98 257 > 258/429,pages2k_3352,ch2k_ZI14TUR01_482,0.0,1.0 ==================================================================== === POTENTIAL DUPLICATE 257/429: pages2k_3352+ch2k_ZI14TUR01_482 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-IfatyTul3.Zinke.2014.txt === === URL 2: https://www.ncdc.noaa.gov/paleo/study/16438 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: True correlation_perfect: True RECORDS IDENTICAL (identical data) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove pages2k_3352, keep ch2k_ZI14TUR01_482. write decision to backup file 1233 4412 98 258 > 259/429,pages2k_3352,iso2k_302,0.0,0.9999999994579302 ==================================================================== === POTENTIAL DUPLICATE 258/429: pages2k_3352+iso2k_302 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-IfatyTul3.Zinke.2014.txt === === URL 2: https://www.ncdc.noaa.gov/paleo/study/16438 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: True RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove pages2k_3352, keep iso2k_302. write decision to backup file 1243 4260 98 259 > 260/429,pages2k_3372,ch2k_KI04MCV01_366,0.0,0.9999999999999999 ==================================================================== === POTENTIAL DUPLICATE 259/429: pages2k_3372+ch2k_KI04MCV01_366 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-MaloChannelEspirituSantoIsland.Kilbourne.2004.txt === === URL 2: https://www.ncdc.noaa.gov/paleo/study/1925 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: True RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove pages2k_3372, keep ch2k_KI04MCV01_366. write decision to backup file 1243 4376 98 260 > 261/429,pages2k_3372,iso2k_155,0.0,0.9999999999999999 ==================================================================== === POTENTIAL DUPLICATE 260/429: pages2k_3372+iso2k_155 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-MaloChannelEspirituSantoIsland.Kilbourne.2004.txt === === URL 2: https://www.ncdc.noaa.gov/paleo/study/1925 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: False data_identical: True correlation_perfect: True RECORDS IDENTICAL (identical data) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove pages2k_3372, keep iso2k_155. write decision to backup file 1244 4261 98 261 > 262/429,pages2k_3374,ch2k_KI04MCV01_368,0.0,1.0 ==================================================================== === POTENTIAL DUPLICATE 261/429: pages2k_3374+ch2k_KI04MCV01_368 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-MaloChannelEspirituSantoIsland.Kilbourne.2004.txt === === URL 2: https://www.ncdc.noaa.gov/paleo/study/1925 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: True RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove pages2k_3374, keep ch2k_KI04MCV01_368. write decision to backup file 1256 1528 98 262 > 263/429,pages2k_3404,FE23_northamerica_canada_cana029,3.9450181369310684,0.9889241082959873 ==================================================================== === POTENTIAL DUPLICATE 262/429: pages2k_3404+FE23_northamerica_canada_cana029 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/NAm-SpruceCreek.Church.1981.txt === === URL 2: https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/northamerica/canada/cana029-noaa.rwl === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) False metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: False Automated choice. Metadata identical, automatically choose FE23 (Breitenmoser et al. (2014)) over PAGES 2k v2.2.0. conservative replication requirement KEEP RED CROSSES: remove pages2k_3404, keep FE23_northamerica_canada_cana029. write decision to backup file 1261 1262 98 263 > 264/429,pages2k_3417,pages2k_3419,0.0,0.9999814351522802 ==================================================================== === POTENTIAL DUPLICATE 263/429: pages2k_3417+pages2k_3419 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Arc-BigRoundLake.Thomas.2009.txt === === URL 2: https://ncei.noaa.gov/pub/data/paleo/paleolimnology/northamerica/canada/baffin/big-round2008.txt === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: False data_identical: False correlation_perfect: True RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #1. KEEP BLUE CIRCLES: keep pages2k_3417, remove pages2k_3419. write decision to backup file 1293 4021 98 264 > 265/429,pages2k_3503,FE23_northamerica_usa_wa072,5.214518434917766,0.9704772757445584 ==================================================================== === POTENTIAL DUPLICATE 264/429: pages2k_3503+FE23_northamerica_usa_wa072 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/NAm-HartsPassN2.Peterson.1994.txt === === URL 2: https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/northamerica/usa/wa072-noaa.rwl === True if pot_dup_corrs[i_pot_dups]>=0.98 else False False (len(time_1)==len(time_2)) False metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: False Automated choice. Metadata identical, automatically choose FE23 (Breitenmoser et al. (2014)) over PAGES 2k v2.2.0. conservative replication requirement KEEP RED CROSSES: remove pages2k_3503, keep FE23_northamerica_usa_wa072. write decision to backup file 1301 2773 98 265 > 266/429,pages2k_3524,FE23_northamerica_usa_ak010,4.098866745195852,0.9844000091013143 ==================================================================== === POTENTIAL DUPLICATE 265/429: pages2k_3524+FE23_northamerica_usa_ak010 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/NAm-EurekaSummit.Schweingruber.1996-1.txt === === URL 2: https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/northamerica/usa/ak010-noaa.rwl === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) False metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: False Automated choice. Metadata identical, automatically choose FE23 (Breitenmoser et al. (2014)) over PAGES 2k v2.2.0. conservative replication requirement KEEP RED CROSSES: remove pages2k_3524, keep FE23_northamerica_usa_ak010. write decision to backup file 1310 2676 98 266 > 267/429,pages2k_3550,FE23_asia_russ137w,0.37072945852236205,0.9906285936546567 ==================================================================== === POTENTIAL DUPLICATE 266/429: pages2k_3550+FE23_asia_russ137w === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Asi-UULEWW.Schweingruber.2002.txt === === URL 2: https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/asia/russ137w-noaa.rwl === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) False metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: False Automated choice. Metadata identical, automatically choose FE23 (Breitenmoser et al. (2014)) over PAGES 2k v2.2.0. conservative replication requirement KEEP RED CROSSES: remove pages2k_3550, keep FE23_asia_russ137w. write decision to backup file 1311 4684 98 267 > 268/429,pages2k_3552,iso2k_1581,0.0,0.9999999999999998 ==================================================================== === POTENTIAL DUPLICATE 267/429: pages2k_3552+iso2k_1581 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-Moorea.Boiseau.1998.txt === === URL 2: https://www.ncdc.noaa.gov/paleo/study/1876 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: True correlation_perfect: True RECORDS IDENTICAL (identical data) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove pages2k_3552, keep iso2k_1581. write decision to backup file 1312 4285 98 268 > 269/429,pages2k_3554,ch2k_LI94SEC01_436,6.617067939395949,1.0 ==================================================================== === POTENTIAL DUPLICATE 268/429: pages2k_3554+ch2k_LI94SEC01_436 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-SecasIslandPanama.Linsley.1994.txt === === URL 2: https://www.ncdc.noaa.gov/paleo/study/1853 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: False data_identical: False correlation_perfect: True RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove pages2k_3554, keep ch2k_LI94SEC01_436. write decision to backup file 1312 4592 98 269 > 270/429,pages2k_3554,iso2k_1124,6.438098381259136,1.0 ==================================================================== === POTENTIAL DUPLICATE 269/429: pages2k_3554+iso2k_1124 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-SecasIslandPanama.Linsley.1994.txt === === URL 2: https://www.ncdc.noaa.gov/paleo/study/1853 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: True correlation_perfect: True RECORDS IDENTICAL (identical data) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove pages2k_3554, keep iso2k_1124. write decision to backup file 1318 4377 98 270 > 271/429,pages2k_3571,iso2k_174,0.0,1.0 ==================================================================== === POTENTIAL DUPLICATE 270/429: pages2k_3571+iso2k_174 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ant-TalosDome.Stenni.2002.txt === === URL 2: https://www.ncdc.noaa.gov/paleo/study/22712 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: True correlation_perfect: True RECORDS IDENTICAL (identical data) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove pages2k_3571, keep iso2k_174. write decision to backup file 1322 3353 98 271 > 272/429,pages2k_3583,FE23_northamerica_usa_co633,6.231818118791423,0.9140277254174102 ==================================================================== === POTENTIAL DUPLICATE 271/429: pages2k_3583+FE23_northamerica_usa_co633 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/NAm-CameronPass.Bigler.2007.txt === === URL 2: https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/northamerica/usa/co633-noaa.rwl === True if pot_dup_corrs[i_pot_dups]>=0.98 else False False (len(time_1)==len(time_2)) False metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: False Automated choice. Metadata identical, automatically choose FE23 (Breitenmoser et al. (2014)) over PAGES 2k v2.2.0. conservative replication requirement KEEP RED CROSSES: remove pages2k_3583, keep FE23_northamerica_usa_co633. write decision to backup file 1328 4582 98 272 > 273/429,pages2k_3599,iso2k_1069,0.0,1.0 ==================================================================== === POTENTIAL DUPLICATE 272/429: pages2k_3599+iso2k_1069 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Arc-AkademiiNaukIceCap.Opel.2013.txt === === URL 2: https://doi.pangaea.de/10.1594/PANGAEA.824732 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: True correlation_perfect: True RECORDS IDENTICAL (identical data) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove pages2k_3599, keep iso2k_1069. write decision to backup file 1328 4701 98 273 > 274/429,pages2k_3599,iso2k_1660,0.0,0.9865701575318525 ==================================================================== === POTENTIAL DUPLICATE 273/429: pages2k_3599+iso2k_1660 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Arc-AkademiiNaukIceCap.Opel.2013.txt === === URL 2: https://doi.pangaea.de/10.1594/PANGAEA.871279 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) False metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: False
**Decision required for this duplicate pair (see figure above).**
---------------------------------------------------------------------------------------------------------
***ATTENTION*** THIS RECORD IS ASSOCIATED WITH MULTIPLE DUPLICATES! PLEASE PAY SPECIAL ATTENTION WHEN MAKING DECISIONS FOR THIS RECORD!
The potential duplicates also associated with this record are:
............................................................
- Dataset ID : iso2k_1069
- URL : https://doi.pangaea.de/10.1594/PANGAEA.824732
---------------------------------------------------------------------------------------------------------
saved figure in /home/jupyter-lluecke/dod2k_v2.0/dod2k/figs//dup_detection/all_merged/273_pages2k_3599_iso2k_1660__1328_4701.jpg KEEP RED CROSSES: remove pages2k_3599, keep iso2k_1660. write decision to backup file 1333 1549 98 274 > 275/429,pages2k_3609,FE23_northamerica_canada_cana053,2.515139756800339,0.9046940174678464 ==================================================================== === POTENTIAL DUPLICATE 274/429: pages2k_3609+FE23_northamerica_canada_cana053 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/NAm-PethaiPeninsula.Schweingruber.1996-1.txt === === URL 2: https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/northamerica/canada/cana053-noaa.rwl === True if pot_dup_corrs[i_pot_dups]>=0.98 else False False (len(time_1)==len(time_2)) False metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: False Automated choice. Metadata identical, automatically choose FE23 (Breitenmoser et al. (2014)) over PAGES 2k v2.2.0. conservative replication requirement KEEP RED CROSSES: remove pages2k_3609, keep FE23_northamerica_canada_cana053. write decision to backup file 1340 4668 98 275 > 276/429,pages2k_3631,iso2k_1530,0.0,1.0 ==================================================================== === POTENTIAL DUPLICATE 275/429: pages2k_3631+iso2k_1530 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-NorthEastBreakersBermuda.Kuhnert.2005-1.txt === === URL 2: http://doi.pangaea.de/10.1594/PANGAEA.738188 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: False data_identical: True correlation_perfect: True RECORDS IDENTICAL (identical data) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove pages2k_3631, keep iso2k_1530. write decision to backup file 1344 4106 98 276 > 277/429,pages2k_3642,FE23_northamerica_usa_wy025,4.425327559545402,0.972318508980092 ==================================================================== === POTENTIAL DUPLICATE 276/429: pages2k_3642+FE23_northamerica_usa_wy025 === === URL 1: https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/NAm-TogwateePass.Briffa.1996-1.txt === === URL 2: https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/northamerica/usa/wy025-noaa.rwl === True if pot_dup_corrs[i_pot_dups]>=0.98 else False False (len(time_1)==len(time_2)) False metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: False Automated choice. Metadata identical, automatically choose FE23 (Breitenmoser et al. (2014)) over PAGES 2k v2.2.0. conservative replication requirement KEEP RED CROSSES: remove pages2k_3642, keep FE23_northamerica_usa_wy025. write decision to backup file 1391 1460 98 277 > 278/429,FE23_southamerica_arge016,FE23_southamerica_arge085,0.0,0.995781123885885 ==================================================================== === POTENTIAL DUPLICATE 277/429: FE23_southamerica_arge016+FE23_southamerica_arge085 === === URL 1: https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/southamerica/arge016-noaa.rwl === === URL 2: https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/southamerica/arge085-noaa.rwl === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: True RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #1. KEEP BLUE CIRCLES: keep FE23_southamerica_arge016, remove FE23_southamerica_arge085. write decision to backup file 1598 1694 98 278 > 279/429,FE23_northamerica_canada_cana100,FE23_northamerica_canada_cana213,0.0,0.9605900887606339 ==================================================================== === POTENTIAL DUPLICATE 278/429: FE23_northamerica_canada_cana100+FE23_northamerica_canada_cana213 === === URL 1: https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/northamerica/canada/cana100-noaa.rwl === === URL 2: https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/northamerica/canada/cana213-noaa.rwl === True if pot_dup_corrs[i_pot_dups]>=0.98 else False False (len(time_1)==len(time_2)) False metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: False Record 2 (FE23_northamerica_canada_cana213) is UPDATE of record 1 (FE23_northamerica_canada_cana100). Automatically choose 2. KEEP RED CROSSES: remove FE23_northamerica_canada_cana100, keep FE23_northamerica_canada_cana213. write decision to backup file 1603 1698 98 279 > 280/429,FE23_northamerica_canada_cana105,FE23_northamerica_canada_cana217,0.0,0.9514191518621167 ==================================================================== === POTENTIAL DUPLICATE 279/429: FE23_northamerica_canada_cana105+FE23_northamerica_canada_cana217 === === URL 1: https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/northamerica/canada/cana105-noaa.rwl === === URL 2: https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/northamerica/canada/cana217-noaa.rwl === True if pot_dup_corrs[i_pot_dups]>=0.98 else False False (len(time_1)==len(time_2)) False metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: False Record 2 (FE23_northamerica_canada_cana217) is UPDATE of record 1 (FE23_northamerica_canada_cana105). Automatically choose 2. KEEP RED CROSSES: remove FE23_northamerica_canada_cana105, keep FE23_northamerica_canada_cana217. write decision to backup file 1612 1649 98 280 > 281/429,FE23_northamerica_canada_cana116,FE23_northamerica_canada_cana168w,0.0,0.9999647944071774 ==================================================================== === POTENTIAL DUPLICATE 280/429: FE23_northamerica_canada_cana116+FE23_northamerica_canada_cana168w === === URL 1: https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/northamerica/canada/cana116-noaa.rwl === === URL 2: https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/northamerica/canada/cana168w-noaa.rwl === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: True RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #1. KEEP BLUE CIRCLES: keep FE23_northamerica_canada_cana116, remove FE23_northamerica_canada_cana168w. write decision to backup file 1647 1648 98 281 > 282/429,FE23_northamerica_canada_cana161,FE23_northamerica_canada_cana162,0.0,0.9987965051468333 ==================================================================== === POTENTIAL DUPLICATE 281/429: FE23_northamerica_canada_cana161+FE23_northamerica_canada_cana162 === === URL 1: https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/northamerica/canada/cana161-noaa.rwl === === URL 2: https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/northamerica/canada/cana162-noaa.rwl === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: False data_identical: False correlation_perfect: True RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #1. KEEP BLUE CIRCLES: keep FE23_northamerica_canada_cana161, remove FE23_northamerica_canada_cana162. write decision to backup file 1795 1796 98 282 > 283/429,FE23_southamerica_chil016,FE23_southamerica_chil017,0.0,0.993020219905921 ==================================================================== === POTENTIAL DUPLICATE 282/429: FE23_southamerica_chil016+FE23_southamerica_chil017 === === URL 1: https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/southamerica/chil016-noaa.rwl === === URL 2: https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/southamerica/chil017-noaa.rwl === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: True RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #1. KEEP BLUE CIRCLES: keep FE23_southamerica_chil016, remove FE23_southamerica_chil017. write decision to backup file 2208 2210 98 283 > 284/429,FE23_europe_swed019w,FE23_europe_swed021w,0.0,0.9940445442585055 ==================================================================== === POTENTIAL DUPLICATE 283/429: FE23_europe_swed019w+FE23_europe_swed021w === === URL 1: https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/europe/swed019w-noaa.rwl === === URL 2: https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/europe/swed021w-noaa.rwl === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) False metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: False
**Decision required for this duplicate pair (see figure above).**
saved figure in /home/jupyter-lluecke/dod2k_v2.0/dod2k/figs//dup_detection/all_merged/283_FE23_europe_swed019w_FE23_europe_swed021w__2208_2210.jpg KEEP BLUE CIRCLES: keep FE23_europe_swed019w, remove FE23_europe_swed021w. write decision to backup file 2388 2389 98 284 > 285/429,FE23_northamerica_mexico_mexi022,FE23_northamerica_mexico_mexi023,0.0,0.9705431503944612 ==================================================================== === POTENTIAL DUPLICATE 284/429: FE23_northamerica_mexico_mexi022+FE23_northamerica_mexico_mexi023 === === URL 1: https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/northamerica/mexico/mexi022-noaa.rwl === === URL 2: https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/northamerica/mexico/mexi023-noaa.rwl === True if pot_dup_corrs[i_pot_dups]>=0.98 else False False (len(time_1)==len(time_2)) False metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: False
**Decision required for this duplicate pair (see figure above).**
saved figure in /home/jupyter-lluecke/dod2k_v2.0/dod2k/figs//dup_detection/all_merged/284_FE23_northamerica_mexico_mexi022_FE23_northamerica_mexico_mexi023__2388_2389.jpg KEEP RED CROSSES: remove FE23_northamerica_mexico_mexi022, keep FE23_northamerica_mexico_mexi023. write decision to backup file 2469 2522 98 285 > 286/429,FE23_australia_newz003,FE23_australia_newz060,0.0,0.9820284222987329 ==================================================================== === POTENTIAL DUPLICATE 285/429: FE23_australia_newz003+FE23_australia_newz060 === === URL 1: https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/australia/newz003-noaa.rwl === === URL 2: https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/australia/newz060-noaa.rwl === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) False metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: False Record 2 (FE23_australia_newz060) is UPDATE of record 1 (FE23_australia_newz003). Automatically choose 2. KEEP RED CROSSES: remove FE23_australia_newz003, keep FE23_australia_newz060. write decision to backup file 2473 2554 98 286 > 287/429,FE23_australia_newz008,FE23_australia_newz092,0.0,0.9876767441282396 ==================================================================== === POTENTIAL DUPLICATE 286/429: FE23_australia_newz008+FE23_australia_newz092 === === URL 1: https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/australia/newz008-noaa.rwl === === URL 2: https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/australia/newz092-noaa.rwl === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) False metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: False data_identical: False correlation_perfect: False
**Decision required for this duplicate pair (see figure above).**
saved figure in /home/jupyter-lluecke/dod2k_v2.0/dod2k/figs//dup_detection/all_merged/286_FE23_australia_newz008_FE23_australia_newz092__2473_2554.jpg KEEP BLUE CIRCLES: keep FE23_australia_newz008, remove FE23_australia_newz092. write decision to backup file 2477 2523 98 287 > 288/429,FE23_australia_newz014,FE23_australia_newz061,0.0,0.9830309576791556 ==================================================================== === POTENTIAL DUPLICATE 287/429: FE23_australia_newz014+FE23_australia_newz061 === === URL 1: https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/australia/newz014-noaa.rwl === === URL 2: https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/australia/newz061-noaa.rwl === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) False metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: False Record 2 (FE23_australia_newz061) is UPDATE of record 1 (FE23_australia_newz014). Automatically choose 2. KEEP RED CROSSES: remove FE23_australia_newz014, keep FE23_australia_newz061. write decision to backup file 2481 2524 98 288 > 289/429,FE23_australia_newz018,FE23_australia_newz062,2.3333445572039433,0.9814682274005635 ==================================================================== === POTENTIAL DUPLICATE 288/429: FE23_australia_newz018+FE23_australia_newz062 === === URL 1: https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/australia/newz018-noaa.rwl === === URL 2: https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/australia/newz062-noaa.rwl === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) False metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: False Record 2 (FE23_australia_newz062) is UPDATE of record 1 (FE23_australia_newz018). Automatically choose 2. KEEP RED CROSSES: remove FE23_australia_newz018, keep FE23_australia_newz062. write decision to backup file 2482 2525 98 289 > 290/429,FE23_australia_newz019,FE23_australia_newz063,0.0,0.9378807954817793 ==================================================================== === POTENTIAL DUPLICATE 289/429: FE23_australia_newz019+FE23_australia_newz063 === === URL 1: https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/australia/newz019-noaa.rwl === === URL 2: https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/australia/newz063-noaa.rwl === True if pot_dup_corrs[i_pot_dups]>=0.98 else False False (len(time_1)==len(time_2)) False metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: False Record 2 (FE23_australia_newz063) is UPDATE of record 1 (FE23_australia_newz019). Automatically choose 2. KEEP RED CROSSES: remove FE23_australia_newz019, keep FE23_australia_newz063. write decision to backup file 3048 3176 98 290 > 291/429,FE23_northamerica_usa_ca066,FE23_northamerica_usa_ca628,0.0,0.913717091970048 ==================================================================== === POTENTIAL DUPLICATE 290/429: FE23_northamerica_usa_ca066+FE23_northamerica_usa_ca628 === === URL 1: https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/northamerica/usa/ca066-noaa.rwl === === URL 2: https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/northamerica/usa/ca628-noaa.rwl === True if pot_dup_corrs[i_pot_dups]>=0.98 else False False (len(time_1)==len(time_2)) False metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: False Record 2 (FE23_northamerica_usa_ca628) is UPDATE of record 1 (FE23_northamerica_usa_ca066). Automatically choose 2. KEEP RED CROSSES: remove FE23_northamerica_usa_ca066, keep FE23_northamerica_usa_ca628. write decision to backup file 3049 3176 98 291 > 292/429,FE23_northamerica_usa_ca067,FE23_northamerica_usa_ca628,0.0,0.9087580435269472 ==================================================================== === POTENTIAL DUPLICATE 291/429: FE23_northamerica_usa_ca067+FE23_northamerica_usa_ca628 === === URL 1: https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/northamerica/usa/ca067-noaa.rwl === === URL 2: https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/northamerica/usa/ca628-noaa.rwl === True if pot_dup_corrs[i_pot_dups]>=0.98 else False False (len(time_1)==len(time_2)) False metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: False Record 2 (FE23_northamerica_usa_ca628) is UPDATE of record 1 (FE23_northamerica_usa_ca067). Automatically choose 2. KEEP RED CROSSES: remove FE23_northamerica_usa_ca067, keep FE23_northamerica_usa_ca628. write decision to backup file 3067 3161 98 292 > 293/429,FE23_northamerica_usa_ca512,FE23_northamerica_usa_ca613,0.0,0.9963133086960106 ==================================================================== === POTENTIAL DUPLICATE 292/429: FE23_northamerica_usa_ca512+FE23_northamerica_usa_ca613 === === URL 1: https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/northamerica/usa/ca512-noaa.rwl === === URL 2: https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/northamerica/usa/ca613-noaa.rwl === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) False metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: False
**Decision required for this duplicate pair (see figure above).**
saved figure in /home/jupyter-lluecke/dod2k_v2.0/dod2k/figs//dup_detection/all_merged/292_FE23_northamerica_usa_ca512_FE23_northamerica_usa_ca613__3067_3161.jpg KEEP RED CROSSES: remove FE23_northamerica_usa_ca512, keep FE23_northamerica_usa_ca613. write decision to backup file 3084 3216 98 293 > 294/429,FE23_northamerica_usa_ca535,FE23_northamerica_usa_ca670,5.5596693510930715,0.9783723385582629 ==================================================================== === POTENTIAL DUPLICATE 293/429: FE23_northamerica_usa_ca535+FE23_northamerica_usa_ca670 === === URL 1: https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/northamerica/usa/ca535-noaa.rwl === === URL 2: https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/northamerica/usa/ca670-noaa.rwl === True if pot_dup_corrs[i_pot_dups]>=0.98 else False False (len(time_1)==len(time_2)) False metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: False Record 2 (FE23_northamerica_usa_ca670) is UPDATE of record 1 (FE23_northamerica_usa_ca535). Automatically choose 2. KEEP RED CROSSES: remove FE23_northamerica_usa_ca535, keep FE23_northamerica_usa_ca670. write decision to backup file 3444 3445 98 294 > 295/429,FE23_northamerica_usa_me017,FE23_northamerica_usa_me018,0.0,0.9519290224263353 ==================================================================== === POTENTIAL DUPLICATE 294/429: FE23_northamerica_usa_me017+FE23_northamerica_usa_me018 === === URL 1: https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/northamerica/usa/me017-noaa.rwl === === URL 2: https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/northamerica/usa/me018-noaa.rwl === True if pot_dup_corrs[i_pot_dups]>=0.98 else False False (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: False
**Decision required for this duplicate pair (see figure above).**
saved figure in /home/jupyter-lluecke/dod2k_v2.0/dod2k/figs//dup_detection/all_merged/294_FE23_northamerica_usa_me017_FE23_northamerica_usa_me018__3444_3445.jpg CREATE A COMPOSITE OF BOTH RECORDS: FE23_northamerica_usa_me017, FE23_northamerica_usa_me018. write decision to backup file 3499 3508 98 295 > 296/429,FE23_northamerica_usa_mo,FE23_northamerica_usa_mo009,0.0,0.9984885968506374 ==================================================================== === POTENTIAL DUPLICATE 295/429: FE23_northamerica_usa_mo+FE23_northamerica_usa_mo009 === === URL 1: https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/northamerica/usa/mo-noaa.rwl === === URL 2: https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/northamerica/usa/mo009-noaa.rwl === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: False lat True lon True elevation False archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: True RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #1. KEEP BLUE CIRCLES: keep FE23_northamerica_usa_mo, remove FE23_northamerica_usa_mo009. write decision to backup file 3547 3548 98 296 > 297/429,FE23_northamerica_usa_mt112,FE23_northamerica_usa_mt113,0.0,0.8683778042327621 ==================================================================== === POTENTIAL DUPLICATE 296/429: FE23_northamerica_usa_mt112+FE23_northamerica_usa_mt113 === === URL 1: https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/northamerica/usa/mt112-noaa.rwl === === URL 2: https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/northamerica/usa/mt113-noaa.rwl === True if pot_dup_corrs[i_pot_dups]>=0.98 else False False (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: False
**Decision required for this duplicate pair (see figure above).** --------------------------------------------------------------------------------------------------------- ***ATTENTION*** THIS RECORD IS ASSOCIATED WITH MULTIPLE DUPLICATES! PLEASE PAY SPECIAL ATTENTION WHEN MAKING DECISIONS FOR THIS RECORD! The potential duplicates also associated with this record are: - pages2k_1089 ---------------------------------------------------------------------------------------------------------
saved figure in /home/jupyter-lluecke/dod2k_v2.0/dod2k/figs//dup_detection/all_merged/296_FE23_northamerica_usa_mt112_FE23_northamerica_usa_mt113__3547_3548.jpg CREATE A COMPOSITE OF BOTH RECORDS: FE23_northamerica_usa_mt112, FE23_northamerica_usa_mt113. write decision to backup file 3588 3589 98 297 > 298/429,FE23_northamerica_usa_nj001,FE23_northamerica_usa_nj002,0.0,0.9931763240446817 ==================================================================== === POTENTIAL DUPLICATE 297/429: FE23_northamerica_usa_nj001+FE23_northamerica_usa_nj002 === === URL 1: https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/northamerica/usa/nj001-noaa.rwl === === URL 2: https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/northamerica/usa/nj002-noaa.rwl === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) False metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: False
**Decision required for this duplicate pair (see figure above).**
saved figure in /home/jupyter-lluecke/dod2k_v2.0/dod2k/figs//dup_detection/all_merged/297_FE23_northamerica_usa_nj001_FE23_northamerica_usa_nj002__3588_3589.jpg CREATE A COMPOSITE OF BOTH RECORDS: FE23_northamerica_usa_nj001, FE23_northamerica_usa_nj002. write decision to backup file 3602 3628 98 298 > 299/429,FE23_northamerica_usa_nm024,FE23_northamerica_usa_nm055,0.0,0.9998373392446366 ==================================================================== === POTENTIAL DUPLICATE 298/429: FE23_northamerica_usa_nm024+FE23_northamerica_usa_nm055 === === URL 1: https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/northamerica/usa/nm024-noaa.rwl === === URL 2: https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/northamerica/usa/nm055-noaa.rwl === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: False data_identical: False correlation_perfect: True RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #1. KEEP BLUE CIRCLES: keep FE23_northamerica_usa_nm024, remove FE23_northamerica_usa_nm055. write decision to backup file 3687 3705 98 299 > 300/429,FE23_northamerica_usa_nv060,FE23_northamerica_usa_nv518,0.0,0.9640681082262826 ==================================================================== === POTENTIAL DUPLICATE 299/429: FE23_northamerica_usa_nv060+FE23_northamerica_usa_nv518 === === URL 1: https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/northamerica/usa/nv060-noaa.rwl === === URL 2: https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/northamerica/usa/nv518-noaa.rwl === True if pot_dup_corrs[i_pot_dups]>=0.98 else False False (len(time_1)==len(time_2)) False metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: False Record 2 (FE23_northamerica_usa_nv518) is UPDATE of record 1 (FE23_northamerica_usa_nv060). Automatically choose 2. KEEP RED CROSSES: remove FE23_northamerica_usa_nv060, keep FE23_northamerica_usa_nv518. write decision to backup file 3699 3708 98 300 > 301/429,FE23_northamerica_usa_nv512,FE23_northamerica_usa_nv521,0.0,0.962931953644384 ==================================================================== === POTENTIAL DUPLICATE 300/429: FE23_northamerica_usa_nv512+FE23_northamerica_usa_nv521 === === URL 1: https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/northamerica/usa/nv512-noaa.rwl === === URL 2: https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/northamerica/usa/nv521-noaa.rwl === True if pot_dup_corrs[i_pot_dups]>=0.98 else False False (len(time_1)==len(time_2)) False metadata_identical: False lat True lon True elevation False archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: False Record 2 (FE23_northamerica_usa_nv521) is UPDATE of record 1 (FE23_northamerica_usa_nv512). Automatically choose 2. KEEP RED CROSSES: remove FE23_northamerica_usa_nv512, keep FE23_northamerica_usa_nv521. write decision to backup file 3700 3707 98 301 > 302/429,FE23_northamerica_usa_nv513,FE23_northamerica_usa_nv520,1.8532231170308289,0.9580289708937026 ==================================================================== === POTENTIAL DUPLICATE 301/429: FE23_northamerica_usa_nv513+FE23_northamerica_usa_nv520 === === URL 1: https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/northamerica/usa/nv513-noaa.rwl === === URL 2: https://www.ncei.noaa.gov/pub/data/paleo/treering/measurements/northamerica/usa/nv520-noaa.rwl === True if pot_dup_corrs[i_pot_dups]>=0.98 else False False (len(time_1)==len(time_2)) False metadata_identical: False lat True lon True elevation False archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: False Record 2 (FE23_northamerica_usa_nv520) is UPDATE of record 1 (FE23_northamerica_usa_nv513). Automatically choose 2. KEEP RED CROSSES: remove FE23_northamerica_usa_nv513, keep FE23_northamerica_usa_nv520. write decision to backup file 4119 4120 98 302 > 303/429,ch2k_ZI15MER01_2,ch2k_ZI15MER01_4,0.0,1.0 ==================================================================== === POTENTIAL DUPLICATE 302/429: ch2k_ZI15MER01_2+ch2k_ZI15MER01_4 === === URL 1: https://www.ncdc.noaa.gov/paleo/study/19239 === === URL 2: https://www.ncdc.noaa.gov/paleo/study/19239 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: True data_identical: True correlation_perfect: True RECORDS IDENTICAL (identical data). Automatically choose #1. KEEP BLUE CIRCLES: keep ch2k_ZI15MER01_2, remove ch2k_ZI15MER01_4. write decision to backup file 4121 4459 98 303 > 304/429,ch2k_CO03PAL03_6,iso2k_511,0.0,1.0 ==================================================================== === POTENTIAL DUPLICATE 303/429: ch2k_CO03PAL03_6+iso2k_511 === === URL 1: https://www.ncdc.noaa.gov/paleo/study/1875 === === URL 2: https://www.ncdc.noaa.gov/paleo/study/1875 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: True data_identical: True correlation_perfect: True RECORDS IDENTICAL (identical data) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove ch2k_CO03PAL03_6, keep iso2k_511. write decision to backup file 4122 4458 98 304 > 305/429,ch2k_CO03PAL02_8,iso2k_509,0.0,1.0 ==================================================================== === POTENTIAL DUPLICATE 304/429: ch2k_CO03PAL02_8+iso2k_509 === === URL 1: https://www.ncdc.noaa.gov/paleo/study/1875 === === URL 2: https://www.ncdc.noaa.gov/paleo/study/1875 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: True data_identical: True correlation_perfect: True RECORDS IDENTICAL (identical data) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove ch2k_CO03PAL02_8, keep iso2k_509. write decision to backup file 4123 4662 98 305 > 306/429,ch2k_LI06RAR01_12,iso2k_1502,0.0,0.9999999999999999 ==================================================================== === POTENTIAL DUPLICATE 305/429: ch2k_LI06RAR01_12+iso2k_1502 === === URL 1: https://www.ncdc.noaa.gov/paleo/study/6089 === === URL 2: https://www.ncdc.noaa.gov/paleo/study/6089 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: True data_identical: False correlation_perfect: True RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove ch2k_LI06RAR01_12, keep iso2k_1502. write decision to backup file 4124 4464 98 306 > 307/429,ch2k_CO03PAL07_14,iso2k_521,0.0,1.0 ==================================================================== === POTENTIAL DUPLICATE 306/429: ch2k_CO03PAL07_14+iso2k_521 === === URL 1: https://www.ncdc.noaa.gov/paleo/study/1875 === === URL 2: https://www.ncdc.noaa.gov/paleo/study/1875 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: True data_identical: True correlation_perfect: True RECORDS IDENTICAL (identical data) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove ch2k_CO03PAL07_14, keep iso2k_521. write decision to backup file 4126 4362 98 307 > 308/429,ch2k_UR00MAI01_22,iso2k_94,7.450067481225913,0.999999993382913 ==================================================================== === POTENTIAL DUPLICATE 307/429: ch2k_UR00MAI01_22+iso2k_94 === === URL 1: https://www.ncdc.noaa.gov/paleo/study/1859 === === URL 2: https://www.ncdc.noaa.gov/paleo/study/1859 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: False lat True lon True elevation False archivetype True paleodata_proxy True sites_identical: False URL_identical: True data_identical: False correlation_perfect: True RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove ch2k_UR00MAI01_22, keep iso2k_94. write decision to backup file 4126 4363 98 308 > 309/429,ch2k_UR00MAI01_22,iso2k_98,7.450067481225913,0.999999993382913 ==================================================================== === POTENTIAL DUPLICATE 308/429: ch2k_UR00MAI01_22+iso2k_98 === === URL 1: https://www.ncdc.noaa.gov/paleo/study/1859 === === URL 2: https://www.ncdc.noaa.gov/paleo/study/1859 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: False lat True lon True elevation False archivetype True paleodata_proxy True sites_identical: False URL_identical: True data_identical: False correlation_perfect: True RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove ch2k_UR00MAI01_22, keep iso2k_98. write decision to backup file 4127 4342 98 309 > 310/429,ch2k_TU95MAD01_24,iso2k_20,0.0,0.9999999999999999 ==================================================================== === POTENTIAL DUPLICATE 309/429: ch2k_TU95MAD01_24+iso2k_20 === === URL 1: https://www.ncdc.noaa.gov/paleo/study/1844 === === URL 2: https://www.ncdc.noaa.gov/paleo/study/1844 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: False lat True lon True elevation False archivetype True paleodata_proxy True sites_identical: False URL_identical: True data_identical: False correlation_perfect: True RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove ch2k_TU95MAD01_24, keep iso2k_20. write decision to backup file 4128 4395 98 310 > 311/429,ch2k_ZI04IFR01_26,iso2k_257,0.0,1.0 ==================================================================== === POTENTIAL DUPLICATE 310/429: ch2k_ZI04IFR01_26+iso2k_257 === === URL 1: https://www.ncdc.noaa.gov/paleo/study/1897 === === URL 2: https://www.ncdc.noaa.gov/paleo/study/16438 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: True RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove ch2k_ZI04IFR01_26, keep iso2k_257. write decision to backup file 4129 4555 98 311 > 312/429,ch2k_RE18CAY01_30,iso2k_917,0.0,1.0 ==================================================================== === POTENTIAL DUPLICATE 311/429: ch2k_RE18CAY01_30+iso2k_917 === === URL 1: https://www.ncdc.noaa.gov/paleo/study/23850 === === URL 2: https://www.ncdc.noaa.gov/paleo/study/23850 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: True data_identical: True correlation_perfect: True RECORDS IDENTICAL (identical data) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove ch2k_RE18CAY01_30, keep iso2k_917. write decision to backup file 4133 4518 98 312 > 313/429,ch2k_KU99HOU01_40,iso2k_786,0.0,0.9999999999999998 ==================================================================== === POTENTIAL DUPLICATE 312/429: ch2k_KU99HOU01_40+iso2k_786 === === URL 1: https://www.ncdc.noaa.gov/paleo/study/1856 === === URL 2: https://www.ncdc.noaa.gov/paleo/study/1856 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: True data_identical: False correlation_perfect: True RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove ch2k_KU99HOU01_40, keep iso2k_786. write decision to backup file 4133 4519 98 313 > 314/429,ch2k_KU99HOU01_40,iso2k_788,0.0,0.9997021095883626 ==================================================================== === POTENTIAL DUPLICATE 313/429: ch2k_KU99HOU01_40+iso2k_788 === === URL 1: https://www.ncdc.noaa.gov/paleo/study/1856 === === URL 2: https://www.ncdc.noaa.gov/paleo/study/1856 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) False metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: True data_identical: False correlation_perfect: False
**Decision required for this duplicate pair (see figure above).**
---------------------------------------------------------------------------------------------------------
***ATTENTION*** THIS RECORD IS ASSOCIATED WITH MULTIPLE DUPLICATES! PLEASE PAY SPECIAL ATTENTION WHEN MAKING DECISIONS FOR THIS RECORD!
The potential duplicates also associated with this record are:
............................................................
- Dataset ID : iso2k_786
- URL : https://www.ncdc.noaa.gov/paleo/study/1856
---------------------------------------------------------------------------------------------------------
saved figure in /home/jupyter-lluecke/dod2k_v2.0/dod2k/figs//dup_detection/all_merged/313_ch2k_KU99HOU01_40_iso2k_788__4133_4519.jpg KEEP BLUE CIRCLES: keep ch2k_KU99HOU01_40, remove iso2k_788. write decision to backup file 4138 4456 98 314 > 315/429,ch2k_NU11PAL01_52,iso2k_505,0.46978588358468626,0.9965291826467186 ==================================================================== === POTENTIAL DUPLICATE 314/429: ch2k_NU11PAL01_52+iso2k_505 === === URL 1: https://www.ncdc.noaa.gov/paleo/study/10373 === === URL 2: https://www.ncdc.noaa.gov/paleo/study/1875 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) False metadata_identical: False lat True lon True elevation False archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: False
**Decision required for this duplicate pair (see figure above).**
---------------------------------------------------------------------------------------------------------
***ATTENTION*** THIS RECORD IS ASSOCIATED WITH MULTIPLE DUPLICATES! PLEASE PAY SPECIAL ATTENTION WHEN MAKING DECISIONS FOR THIS RECORD!
The potential duplicates also associated with this record are:
- pages2k_1488
............................................................
- Dataset ID : pages2k_1628
- URL : https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-Palmyra.Cobb.2003.txt
............................................................
- Dataset ID : iso2k_579
- URL : https://www.ncdc.noaa.gov/paleo/study/10373
---------------------------------------------------------------------------------------------------------
saved figure in /home/jupyter-lluecke/dod2k_v2.0/dod2k/figs//dup_detection/all_merged/314_ch2k_NU11PAL01_52_iso2k_505__4138_4456.jpg KEEP BOTH: keep ch2k_NU11PAL01_52, keep iso2k_505. write decision to backup file 4138 4482 98 315 > 316/429,ch2k_NU11PAL01_52,iso2k_579,0.46978588358468626,0.9992710430546083 ==================================================================== === POTENTIAL DUPLICATE 315/429: ch2k_NU11PAL01_52+iso2k_579 === === URL 1: https://www.ncdc.noaa.gov/paleo/study/10373 === === URL 2: https://www.ncdc.noaa.gov/paleo/study/10373 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: True data_identical: False correlation_perfect: True RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove ch2k_NU11PAL01_52, keep iso2k_579. write decision to backup file 4141 4445 98 316 > 317/429,ch2k_CA14TIM01_64,iso2k_473,0.7330803840277191,1.0 ==================================================================== === POTENTIAL DUPLICATE 316/429: ch2k_CA14TIM01_64+iso2k_473 === === URL 1: https://www.ncdc.noaa.gov/paleo/study/19179 === === URL 2: https://www.ncdc.noaa.gov/paleo/study/19179 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: True data_identical: True correlation_perfect: True RECORDS IDENTICAL (identical data) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove ch2k_CA14TIM01_64, keep iso2k_473. write decision to backup file 4146 4736 98 317 > 318/429,ch2k_HE08LRA01_76,iso2k_1813,0.0,1.0 ==================================================================== === POTENTIAL DUPLICATE 317/429: ch2k_HE08LRA01_76+iso2k_1813 === === URL 1: https://www.ncdc.noaa.gov/paleo/study/12891 === === URL 2: https://www.ncdc.noaa.gov/paleo/study/12891 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: True data_identical: True correlation_perfect: True RECORDS IDENTICAL (identical data). Automatically choose #2. KEEP RED CROSSES: remove ch2k_HE08LRA01_76, keep iso2k_1813. write decision to backup file 4147 4719 98 318 > 319/429,ch2k_DA06MAF01_78,iso2k_1748,0.0,1.0 ==================================================================== === POTENTIAL DUPLICATE 318/429: ch2k_DA06MAF01_78+iso2k_1748 === === URL 1: https://www.ncdc.noaa.gov/paleo/study/10808 === === URL 2: https://www.ncdc.noaa.gov/paleo/study/10808 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) False metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: True data_identical: False correlation_perfect: False
**Decision required for this duplicate pair (see figure above).**
---------------------------------------------------------------------------------------------------------
***ATTENTION*** THIS RECORD IS ASSOCIATED WITH MULTIPLE DUPLICATES! PLEASE PAY SPECIAL ATTENTION WHEN MAKING DECISIONS FOR THIS RECORD!
The potential duplicates also associated with this record are:
- pages2k_1147
............................................................
- Dataset ID : ch2k_DA06MAF02_104
- URL : https://www.ncdc.noaa.gov/paleo/study/10808
---------------------------------------------------------------------------------------------------------
saved figure in /home/jupyter-lluecke/dod2k_v2.0/dod2k/figs//dup_detection/all_merged/318_ch2k_DA06MAF01_78_iso2k_1748__4147_4719.jpg KEEP RED CROSSES: remove ch2k_DA06MAF01_78, keep iso2k_1748. write decision to backup file 4148 4722 98 319 > 320/429,ch2k_NA09MAL01_84,iso2k_1754,0.0,0.9999999999999999 ==================================================================== === POTENTIAL DUPLICATE 319/429: ch2k_NA09MAL01_84+iso2k_1754 === === URL 1: https://www.ncdc.noaa.gov/paleo/study/12994 === === URL 2: https://www.ncdc.noaa.gov/paleo/study/12994 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: False lat True lon True elevation False archivetype True paleodata_proxy True sites_identical: False URL_identical: True data_identical: False correlation_perfect: True RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove ch2k_NA09MAL01_84, keep iso2k_1754. write decision to backup file 4149 4349 98 320 > 321/429,ch2k_SW98STP01_86,iso2k_50,0.5188380044682307,1.0 ==================================================================== === POTENTIAL DUPLICATE 320/429: ch2k_SW98STP01_86+iso2k_50 === === URL 1: https://www.ncdc.noaa.gov/paleo/study/1913 === === URL 2: https://www.ncdc.noaa.gov/paleo/study/1913 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: True data_identical: False correlation_perfect: True RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove ch2k_SW98STP01_86, keep iso2k_50. write decision to backup file 4153 4719 98 321 > 322/429,ch2k_DA06MAF02_104,iso2k_1748,0.0,1.0 ==================================================================== === POTENTIAL DUPLICATE 321/429: ch2k_DA06MAF02_104+iso2k_1748 === === URL 1: https://www.ncdc.noaa.gov/paleo/study/10808 === === URL 2: https://www.ncdc.noaa.gov/paleo/study/10808 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) False metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: True data_identical: False correlation_perfect: False
**Decision required for this duplicate pair (see figure above).**
---------------------------------------------------------------------------------------------------------
***ATTENTION*** THIS RECORD IS ASSOCIATED WITH MULTIPLE DUPLICATES! PLEASE PAY SPECIAL ATTENTION WHEN MAKING DECISIONS FOR THIS RECORD!
The potential duplicates also associated with this record are:
- pages2k_1147
............................................................
- Dataset ID : ch2k_DA06MAF01_78
- URL : https://www.ncdc.noaa.gov/paleo/study/10808
---------------------------------------------------------------------------------------------------------
saved figure in /home/jupyter-lluecke/dod2k_v2.0/dod2k/figs//dup_detection/all_merged/321_ch2k_DA06MAF02_104_iso2k_1748__4153_4719.jpg KEEP RED CROSSES: remove ch2k_DA06MAF02_104, keep iso2k_1748. write decision to backup file 4156 4457 98 322 > 323/429,ch2k_CO03PAL01_110,iso2k_507,0.0,1.0 ==================================================================== === POTENTIAL DUPLICATE 322/429: ch2k_CO03PAL01_110+iso2k_507 === === URL 1: https://www.ncdc.noaa.gov/paleo/study/1875 === === URL 2: https://www.ncdc.noaa.gov/paleo/study/1875 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: True data_identical: True correlation_perfect: True RECORDS IDENTICAL (identical data) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove ch2k_CO03PAL01_110, keep iso2k_507. write decision to backup file 4159 4611 98 323 > 324/429,ch2k_CH98PIR01_116,iso2k_1229,0.0,1.0 ==================================================================== === POTENTIAL DUPLICATE 323/429: ch2k_CH98PIR01_116+iso2k_1229 === === URL 1: https://www.ncdc.noaa.gov/paleo/study/1901 === === URL 2: https://www.ncdc.noaa.gov/paleo/study/1901 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: True data_identical: True correlation_perfect: True RECORDS IDENTICAL (identical data) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove ch2k_CH98PIR01_116, keep iso2k_1229. write decision to backup file 4164 4167 98 324 > 325/429,ch2k_XI17HAI01_128,ch2k_XI17HAI01_136,0.0,1.0 ==================================================================== === POTENTIAL DUPLICATE 324/429: ch2k_XI17HAI01_128+ch2k_XI17HAI01_136 === === URL 1: https://www.ncdc.noaa.gov/paleo/study/34452 === === URL 2: https://www.ncdc.noaa.gov/paleo/study/34452 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: True data_identical: True correlation_perfect: True RECORDS IDENTICAL (identical data). Automatically choose #1. KEEP BLUE CIRCLES: keep ch2k_XI17HAI01_128, remove ch2k_XI17HAI01_136. write decision to backup file 4164 4724 98 325 > 326/429,ch2k_XI17HAI01_128,iso2k_1762,0.010402712124816737,1.0 ==================================================================== === POTENTIAL DUPLICATE 325/429: ch2k_XI17HAI01_128+iso2k_1762 === === URL 1: https://www.ncdc.noaa.gov/paleo/study/34452 === === URL 2: https://ars-els-cdn-com.libezp.lib.lsu.edu/content/image/1-s2.0-S1367912016304138-mmc1.xlsx === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: True RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove ch2k_XI17HAI01_128, keep iso2k_1762. write decision to backup file 4165 4166 98 326 > 327/429,ch2k_XI17HAI01_130,ch2k_XI17HAI01_134,0.0,0.9999999999999999 ==================================================================== === POTENTIAL DUPLICATE 326/429: ch2k_XI17HAI01_130+ch2k_XI17HAI01_134 === === URL 1: https://www.ncdc.noaa.gov/paleo/study/34452 === === URL 2: https://www.ncdc.noaa.gov/paleo/study/34452 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: True data_identical: True correlation_perfect: True RECORDS IDENTICAL (identical data). Automatically choose #1. KEEP BLUE CIRCLES: keep ch2k_XI17HAI01_130, remove ch2k_XI17HAI01_134. write decision to backup file 4167 4724 98 327 > 328/429,ch2k_XI17HAI01_136,iso2k_1762,0.010402712124816737,1.0 ==================================================================== === POTENTIAL DUPLICATE 327/429: ch2k_XI17HAI01_136+iso2k_1762 === === URL 1: https://www.ncdc.noaa.gov/paleo/study/34452 === === URL 2: https://ars-els-cdn-com.libezp.lib.lsu.edu/content/image/1-s2.0-S1367912016304138-mmc1.xlsx === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: True RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove ch2k_XI17HAI01_136, keep iso2k_1762. write decision to backup file 4168 4172 98 328 > 329/429,ch2k_DE14DTO03_140,ch2k_DE14DTO01_148,0.0,0.9484736035921149 ==================================================================== === POTENTIAL DUPLICATE 328/429: ch2k_DE14DTO03_140+ch2k_DE14DTO01_148 === === URL 1: https://www.ncdc.noaa.gov/paleo/study/11935 === === URL 2: https://www.ncdc.noaa.gov/paleo/study/16217 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False False (len(time_1)==len(time_2)) False metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: False data_identical: False correlation_perfect: False
**Decision required for this duplicate pair (see figure above).**
saved figure in /home/jupyter-lluecke/dod2k_v2.0/dod2k/figs//dup_detection/all_merged/328_ch2k_DE14DTO03_140_ch2k_DE14DTO01_148__4168_4172.jpg KEEP BOTH: keep ch2k_DE14DTO03_140, keep ch2k_DE14DTO01_148. write decision to backup file 4170 4626 98 329 > 330/429,ch2k_QU06RAB01_144,iso2k_1311,1.3265164823623186,0.9998938624989858 ==================================================================== === POTENTIAL DUPLICATE 329/429: ch2k_QU06RAB01_144+iso2k_1311 === === URL 1: https://www.ncdc.noaa.gov/paleo/study/6116 === === URL 2: https://www.ncdc.noaa.gov/paleo/study/6116 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: True data_identical: False correlation_perfect: True RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove ch2k_QU06RAB01_144, keep iso2k_1311. write decision to backup file 4173 4672 98 330 > 331/429,ch2k_KU00NIN01_150,iso2k_1554,0.0,1.0 ==================================================================== === POTENTIAL DUPLICATE 330/429: ch2k_KU00NIN01_150+iso2k_1554 === === URL 1: https://www.ncdc.noaa.gov/paleo/study/1867 === === URL 2: https://www.ncdc.noaa.gov/paleo/study/1867 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: True data_identical: False correlation_perfect: True RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove ch2k_KU00NIN01_150, keep iso2k_1554. write decision to backup file 4173 4673 98 331 > 332/429,ch2k_KU00NIN01_150,iso2k_1556,0.0,0.9995653924115724 ==================================================================== === POTENTIAL DUPLICATE 331/429: ch2k_KU00NIN01_150+iso2k_1556 === === URL 1: https://www.ncdc.noaa.gov/paleo/study/1867 === === URL 2: https://www.ncdc.noaa.gov/paleo/study/1867 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) False metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: True data_identical: False correlation_perfect: False
**Decision required for this duplicate pair (see figure above).**
---------------------------------------------------------------------------------------------------------
***ATTENTION*** THIS RECORD IS ASSOCIATED WITH MULTIPLE DUPLICATES! PLEASE PAY SPECIAL ATTENTION WHEN MAKING DECISIONS FOR THIS RECORD!
The potential duplicates also associated with this record are:
- pages2k_3085
............................................................
- Dataset ID : iso2k_1554
- URL : https://www.ncdc.noaa.gov/paleo/study/1867
---------------------------------------------------------------------------------------------------------
saved figure in /home/jupyter-lluecke/dod2k_v2.0/dod2k/figs//dup_detection/all_merged/331_ch2k_KU00NIN01_150_iso2k_1556__4173_4673.jpg KEEP BLUE CIRCLES: keep ch2k_KU00NIN01_150, remove iso2k_1556. write decision to backup file 4187 4188 98 332 > 333/429,ch2k_EV18ROC01_184,ch2k_EV18ROC01_186,0.0,1.0 ==================================================================== === POTENTIAL DUPLICATE 332/429: ch2k_EV18ROC01_184+ch2k_EV18ROC01_186 === === URL 1: https://www.ncdc.noaa.gov/paleo/study/34373 === === URL 2: https://www.ncdc.noaa.gov/paleo/study/34373 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: True data_identical: True correlation_perfect: True RECORDS IDENTICAL (identical data). Automatically choose #1. KEEP BLUE CIRCLES: keep ch2k_EV18ROC01_184, remove ch2k_EV18ROC01_186. write decision to backup file 4189 4478 98 333 > 334/429,ch2k_CA13SAP01_188,iso2k_569,0.038808523132686956,1.0 ==================================================================== === POTENTIAL DUPLICATE 333/429: ch2k_CA13SAP01_188+iso2k_569 === === URL 1: https://www.ncdc.noaa.gov/paleo/study/17378 === === URL 2: https://www.ncdc.noaa.gov/paleo/study/17378 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: True data_identical: True correlation_perfect: True RECORDS IDENTICAL (identical data) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove ch2k_CA13SAP01_188, keep iso2k_569. write decision to backup file 4191 4383 98 334 > 335/429,ch2k_HE13MIS01_194,iso2k_211,0.18098363234620218,0.9277960229464544 ==================================================================== === POTENTIAL DUPLICATE 334/429: ch2k_HE13MIS01_194+iso2k_211 === === URL 1: https://www.ncdc.noaa.gov/paleo/study/15794 === === URL 2: https://www.ncdc.noaa.gov/paleo/study/15794 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False False (len(time_1)==len(time_2)) False metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: True data_identical: False correlation_perfect: False
**Decision required for this duplicate pair (see figure above).**
---------------------------------------------------------------------------------------------------------
***ATTENTION*** THIS RECORD IS ASSOCIATED WITH MULTIPLE DUPLICATES! PLEASE PAY SPECIAL ATTENTION WHEN MAKING DECISIONS FOR THIS RECORD!
The potential duplicates also associated with this record are:
............................................................
- Dataset ID : iso2k_213
- URL : https://www.ncdc.noaa.gov/paleo/study/15794
---------------------------------------------------------------------------------------------------------
saved figure in /home/jupyter-lluecke/dod2k_v2.0/dod2k/figs//dup_detection/all_merged/334_ch2k_HE13MIS01_194_iso2k_211__4191_4383.jpg KEEP BOTH: keep ch2k_HE13MIS01_194, keep iso2k_211. write decision to backup file 4191 4384 98 335 > 336/429,ch2k_HE13MIS01_194,iso2k_213,0.18098363234620218,0.993099103724861 ==================================================================== === POTENTIAL DUPLICATE 335/429: ch2k_HE13MIS01_194+iso2k_213 === === URL 1: https://www.ncdc.noaa.gov/paleo/study/15794 === === URL 2: https://www.ncdc.noaa.gov/paleo/study/15794 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) False metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: True data_identical: False correlation_perfect: False
**Decision required for this duplicate pair (see figure above).**
---------------------------------------------------------------------------------------------------------
***ATTENTION*** THIS RECORD IS ASSOCIATED WITH MULTIPLE DUPLICATES! PLEASE PAY SPECIAL ATTENTION WHEN MAKING DECISIONS FOR THIS RECORD!
The potential duplicates also associated with this record are:
............................................................
- Dataset ID : iso2k_211
- URL : https://www.ncdc.noaa.gov/paleo/study/15794
---------------------------------------------------------------------------------------------------------
saved figure in /home/jupyter-lluecke/dod2k_v2.0/dod2k/figs//dup_detection/all_merged/335_ch2k_HE13MIS01_194_iso2k_213__4191_4384.jpg KEEP BLUE CIRCLES: keep ch2k_HE13MIS01_194, remove iso2k_213. write decision to backup file 4193 4194 98 336 > 337/429,ch2k_ZI15IMP02_200,ch2k_ZI15IMP02_202,0.0,1.0 ==================================================================== === POTENTIAL DUPLICATE 336/429: ch2k_ZI15IMP02_200+ch2k_ZI15IMP02_202 === === URL 1: https://www.ncdc.noaa.gov/paleo/study/19239 === === URL 2: https://www.ncdc.noaa.gov/paleo/study/19239 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: True data_identical: True correlation_perfect: True RECORDS IDENTICAL (identical data). Automatically choose #1. KEEP BLUE CIRCLES: keep ch2k_ZI15IMP02_200, remove ch2k_ZI15IMP02_202. write decision to backup file 4195 4705 98 337 > 338/429,ch2k_PF04PBA01_204,iso2k_1701,0.0,1.0 ==================================================================== === POTENTIAL DUPLICATE 337/429: ch2k_PF04PBA01_204+iso2k_1701 === === URL 1: https://www.ncdc.noaa.gov/paleo/study/1891 === === URL 2: https://www.ncdc.noaa.gov/paleo/study/1891 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: True data_identical: False correlation_perfect: True RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove ch2k_PF04PBA01_204, keep iso2k_1701. write decision to backup file 4195 4706 98 338 > 339/429,ch2k_PF04PBA01_204,iso2k_1704,0.0,1.0 ==================================================================== === POTENTIAL DUPLICATE 338/429: ch2k_PF04PBA01_204+iso2k_1704 === === URL 1: https://www.ncdc.noaa.gov/paleo/study/1891 === === URL 2: https://www.ncdc.noaa.gov/paleo/study/1891 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: True data_identical: False correlation_perfect: True RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove ch2k_PF04PBA01_204, keep iso2k_1704. write decision to backup file 4199 4461 98 339 > 340/429,ch2k_CO03PAL05_212,iso2k_515,0.0,1.0 ==================================================================== === POTENTIAL DUPLICATE 339/429: ch2k_CO03PAL05_212+iso2k_515 === === URL 1: https://www.ncdc.noaa.gov/paleo/study/1875 === === URL 2: https://www.ncdc.noaa.gov/paleo/study/1875 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: True data_identical: True correlation_perfect: True RECORDS IDENTICAL (identical data) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove ch2k_CO03PAL05_212, keep iso2k_515. write decision to backup file 4204 4490 98 340 > 341/429,ch2k_MO06PED01_226,iso2k_629,1.126602565892204,1.0 ==================================================================== === POTENTIAL DUPLICATE 340/429: ch2k_MO06PED01_226+iso2k_629 === === URL 1: https://www.ncdc.noaa.gov/paleo/study/6184 === === URL 2: https://www.ncdc.noaa.gov/paleo/study/6184 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: True data_identical: False correlation_perfect: True RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove ch2k_MO06PED01_226, keep iso2k_629. write decision to backup file 4208 4422 98 341 > 342/429,ch2k_OS14UCP01_236,iso2k_350,0.0,1.0 ==================================================================== === POTENTIAL DUPLICATE 341/429: ch2k_OS14UCP01_236+iso2k_350 === === URL 1: https://www.ncdc.noaa.gov/paleo/study/16339 === === URL 2: https://www.ncdc.noaa.gov/paleo/study/16339 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: True data_identical: False correlation_perfect: True RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove ch2k_OS14UCP01_236, keep iso2k_350. write decision to backup file 4212 4715 98 342 > 343/429,ch2k_HE10GUA01_244,iso2k_1735,0.0,0.9999993709991193 ==================================================================== === POTENTIAL DUPLICATE 342/429: ch2k_HE10GUA01_244+iso2k_1735 === === URL 1: https://www.ncdc.noaa.gov/paleo/study/12893 === === URL 2: https://www.ncdc.noaa.gov/paleo/study/12893 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: True data_identical: False correlation_perfect: True RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove ch2k_HE10GUA01_244, keep iso2k_1735. write decision to backup file 4219 4220 98 343 > 344/429,ch2k_DR99ABR01_264,ch2k_DR99ABR01_266,0.0,1.0 ==================================================================== === POTENTIAL DUPLICATE 343/429: ch2k_DR99ABR01_264+ch2k_DR99ABR01_266 === === URL 1: https://www.ncdc.noaa.gov/paleo/study/1911 === === URL 2: https://www.ncdc.noaa.gov/paleo/study/1911 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: True data_identical: True correlation_perfect: True RECORDS IDENTICAL (identical data). Automatically choose #1. KEEP BLUE CIRCLES: keep ch2k_DR99ABR01_264, remove ch2k_DR99ABR01_266. write decision to backup file 4219 4361 98 344 > 345/429,ch2k_DR99ABR01_264,iso2k_91,0.0,1.0 ==================================================================== === POTENTIAL DUPLICATE 344/429: ch2k_DR99ABR01_264+iso2k_91 === === URL 1: https://www.ncdc.noaa.gov/paleo/study/1911 === === URL 2: https://www.ncdc.noaa.gov/paleo/study/1911 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: True data_identical: True correlation_perfect: True RECORDS IDENTICAL (identical data) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove ch2k_DR99ABR01_264, keep iso2k_91. write decision to backup file 4220 4361 98 345 > 346/429,ch2k_DR99ABR01_266,iso2k_91,0.0,1.0 ==================================================================== === POTENTIAL DUPLICATE 345/429: ch2k_DR99ABR01_266+iso2k_91 === === URL 1: https://www.ncdc.noaa.gov/paleo/study/1911 === === URL 2: https://www.ncdc.noaa.gov/paleo/study/1911 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: True data_identical: True correlation_perfect: True RECORDS IDENTICAL (identical data) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove ch2k_DR99ABR01_266, keep iso2k_91. write decision to backup file 4221 4661 98 346 > 347/429,ch2k_LI06RAR02_270,iso2k_1500,0.0,1.0 ==================================================================== === POTENTIAL DUPLICATE 346/429: ch2k_LI06RAR02_270+iso2k_1500 === === URL 1: https://www.ncdc.noaa.gov/paleo/study/6089 === === URL 2: https://www.ncdc.noaa.gov/paleo/study/6089 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: False lat True lon True elevation False archivetype True paleodata_proxy True sites_identical: False URL_identical: True data_identical: False correlation_perfect: True RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove ch2k_LI06RAR02_270, keep iso2k_1500. write decision to backup file 4225 4226 98 347 > 348/429,ch2k_ZI15TAN01_278,ch2k_ZI15TAN01_280,0.0,1.0 ==================================================================== === POTENTIAL DUPLICATE 347/429: ch2k_ZI15TAN01_278+ch2k_ZI15TAN01_280 === === URL 1: https://www.ncdc.noaa.gov/paleo/study/19239 === === URL 2: https://www.ncdc.noaa.gov/paleo/study/19239 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: True data_identical: True correlation_perfect: True RECORDS IDENTICAL (identical data). Automatically choose #1. KEEP BLUE CIRCLES: keep ch2k_ZI15TAN01_278, remove ch2k_ZI15TAN01_280. write decision to backup file 4235 4675 98 348 > 349/429,ch2k_AS05GUA01_302,iso2k_1559,0.0,1.0 ==================================================================== === POTENTIAL DUPLICATE 348/429: ch2k_AS05GUA01_302+iso2k_1559 === === URL 1: https://www.ncdc.noaa.gov/paleo/study/1915 === === URL 2: https://www.ncdc.noaa.gov/paleo/study/1915 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: True data_identical: False correlation_perfect: True RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove ch2k_AS05GUA01_302, keep iso2k_1559. write decision to backup file 4236 4769 98 349 > 350/429,ch2k_FE09OGA01_304,iso2k_1922,0.0,1.0 ==================================================================== === POTENTIAL DUPLICATE 349/429: ch2k_FE09OGA01_304+iso2k_1922 === === URL 1: https://doi.pangaea.de/10.1594/PANGAEA.743953 === === URL 2: https://www.ncdc.noaa.gov/paleo/study/8608 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: False lat True lon True elevation False archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: True RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove ch2k_FE09OGA01_304, keep iso2k_1922. write decision to backup file 4240 4501 98 350 > 351/429,ch2k_GU99NAU01_314,iso2k_702,4.70176061145538,0.9755996990643612 ==================================================================== === POTENTIAL DUPLICATE 350/429: ch2k_GU99NAU01_314+iso2k_702 === === URL 1: https://www.ncdc.noaa.gov/paleo/study/1842 === === URL 2: https://www.ncdc.noaa.gov/paleo/study/1842 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False False (len(time_1)==len(time_2)) False metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: True data_identical: False correlation_perfect: False
**Decision required for this duplicate pair (see figure above).**
---------------------------------------------------------------------------------------------------------
***ATTENTION*** THIS RECORD IS ASSOCIATED WITH MULTIPLE DUPLICATES! PLEASE PAY SPECIAL ATTENTION WHEN MAKING DECISIONS FOR THIS RECORD!
The potential duplicates also associated with this record are:
............................................................
- Dataset ID : iso2k_705
- URL : https://www.ncdc.noaa.gov/paleo/study/1842
---------------------------------------------------------------------------------------------------------
saved figure in /home/jupyter-lluecke/dod2k_v2.0/dod2k/figs//dup_detection/all_merged/350_ch2k_GU99NAU01_314_iso2k_702__4240_4501.jpg KEEP BLUE CIRCLES: keep ch2k_GU99NAU01_314, remove iso2k_702. write decision to backup file 4240 4502 98 351 > 352/429,ch2k_GU99NAU01_314,iso2k_705,4.70176061145538,1.0 ==================================================================== === POTENTIAL DUPLICATE 351/429: ch2k_GU99NAU01_314+iso2k_705 === === URL 1: https://www.ncdc.noaa.gov/paleo/study/1842 === === URL 2: https://www.ncdc.noaa.gov/paleo/study/1842 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: True data_identical: True correlation_perfect: True RECORDS IDENTICAL (identical data) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove ch2k_GU99NAU01_314, keep iso2k_705. write decision to backup file 4243 4463 98 352 > 353/429,ch2k_CO03PAL10_324,iso2k_519,0.0,1.0 ==================================================================== === POTENTIAL DUPLICATE 352/429: ch2k_CO03PAL10_324+iso2k_519 === === URL 1: https://www.ncdc.noaa.gov/paleo/study/1875 === === URL 2: https://www.ncdc.noaa.gov/paleo/study/1875 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: True data_identical: True correlation_perfect: True RECORDS IDENTICAL (identical data) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove ch2k_CO03PAL10_324, keep iso2k_519. write decision to backup file 4245 4246 98 353 > 354/429,ch2k_ZI15IMP01_328,ch2k_ZI15IMP01_330,0.0,1.0 ==================================================================== === POTENTIAL DUPLICATE 353/429: ch2k_ZI15IMP01_328+ch2k_ZI15IMP01_330 === === URL 1: https://www.ncdc.noaa.gov/paleo/study/19239 === === URL 2: https://www.ncdc.noaa.gov/paleo/study/19239 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: True data_identical: True correlation_perfect: True RECORDS IDENTICAL (identical data). Automatically choose #1. KEEP BLUE CIRCLES: keep ch2k_ZI15IMP01_328, remove ch2k_ZI15IMP01_330. write decision to backup file 4249 4250 98 354 > 355/429,ch2k_RO19YUC01_338,ch2k_RO19YUC01_340,0.0,1.0 ==================================================================== === POTENTIAL DUPLICATE 354/429: ch2k_RO19YUC01_338+ch2k_RO19YUC01_340 === === URL 1: https://www.ncdc.noaa.gov/paleo/study/27450 === === URL 2: https://www.ncdc.noaa.gov/paleo/study/27450 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: True data_identical: True correlation_perfect: True RECORDS IDENTICAL (identical data). Automatically choose #1. KEEP BLUE CIRCLES: keep ch2k_RO19YUC01_338, remove ch2k_RO19YUC01_340. write decision to backup file 4257 4466 98 355 > 356/429,ch2k_CO03PAL09_358,iso2k_525,0.0,1.0 ==================================================================== === POTENTIAL DUPLICATE 355/429: ch2k_CO03PAL09_358+iso2k_525 === === URL 1: https://www.ncdc.noaa.gov/paleo/study/1875 === === URL 2: https://www.ncdc.noaa.gov/paleo/study/1875 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: True data_identical: True correlation_perfect: True RECORDS IDENTICAL (identical data) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove ch2k_CO03PAL09_358, keep iso2k_525. write decision to backup file 4260 4376 98 356 > 357/429,ch2k_KI04MCV01_366,iso2k_155,0.0,0.9999999999999999 ==================================================================== === POTENTIAL DUPLICATE 356/429: ch2k_KI04MCV01_366+iso2k_155 === === URL 1: https://www.ncdc.noaa.gov/paleo/study/1925 === === URL 2: https://www.ncdc.noaa.gov/paleo/study/1925 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: True data_identical: False correlation_perfect: True RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove ch2k_KI04MCV01_366, keep iso2k_155. write decision to backup file 4264 4350 98 357 > 358/429,ch2k_BA04FIJ02_382,iso2k_52,0.5087587517215614,1.0 ==================================================================== === POTENTIAL DUPLICATE 357/429: ch2k_BA04FIJ02_382+iso2k_52 === === URL 1: https://www.ncdc.noaa.gov/paleo/study/1881 === === URL 2: https://www.ncdc.noaa.gov/paleo/study/1881 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: False lat True lon True elevation False archivetype True paleodata_proxy True sites_identical: False URL_identical: True data_identical: False correlation_perfect: True RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove ch2k_BA04FIJ02_382, keep iso2k_52. write decision to backup file 4265 4462 98 358 > 359/429,ch2k_CO03PAL06_386,iso2k_517,0.0,1.0 ==================================================================== === POTENTIAL DUPLICATE 358/429: ch2k_CO03PAL06_386+iso2k_517 === === URL 1: https://www.ncdc.noaa.gov/paleo/study/1875 === === URL 2: https://www.ncdc.noaa.gov/paleo/study/1875 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: True data_identical: True correlation_perfect: True RECORDS IDENTICAL (identical data) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove ch2k_CO03PAL06_386, keep iso2k_517. write decision to backup file 4269 4538 98 359 > 360/429,ch2k_GO12SBV01_396,iso2k_870,3.209089318097252,0.9999273240076922 ==================================================================== === POTENTIAL DUPLICATE 359/429: ch2k_GO12SBV01_396+iso2k_870 === === URL 1: https://www.ncdc.noaa.gov/paleo/study/13439 === === URL 2: https://www.ncdc.noaa.gov/paleo/study/13439 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: True data_identical: False correlation_perfect: True RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove ch2k_GO12SBV01_396, keep iso2k_870. write decision to backup file 4271 4579 98 360 > 361/429,ch2k_CA07FLI01_400,iso2k_1057,0.0,0.9999999999999998 ==================================================================== === POTENTIAL DUPLICATE 360/429: ch2k_CA07FLI01_400+iso2k_1057 === === URL 1: https://www.ncdc.noaa.gov/paleo/study/6087 === === URL 2: https://www.ncdc.noaa.gov/paleo/study/6087 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: True data_identical: False correlation_perfect: True RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove ch2k_CA07FLI01_400, keep iso2k_1057. write decision to backup file 4274 4469 98 361 > 362/429,ch2k_CO93TAR01_408,iso2k_539,0.5185313171246082,1.0 ==================================================================== === POTENTIAL DUPLICATE 361/429: ch2k_CO93TAR01_408+iso2k_539 === === URL 1: https://www.ncdc.noaa.gov/paleo/study/1845 === === URL 2: https://www.ncdc.noaa.gov/paleo/study/1845 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: True data_identical: False correlation_perfect: True RECORDS IDENTICAL (perfect correlation). Automatically choose #2. KEEP RED CROSSES: remove ch2k_CO93TAR01_408, keep iso2k_539. write decision to backup file 4276 4570 98 362 > 363/429,ch2k_CO00MAL01_412,iso2k_1010,0.611199580068607,0.9999999999999999 ==================================================================== === POTENTIAL DUPLICATE 362/429: ch2k_CO00MAL01_412+iso2k_1010 === === URL 1: https://www.ncdc.noaa.gov/paleo/study/1855 === === URL 2: https://www.ncdc.noaa.gov/paleo/study/1855 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: True data_identical: False correlation_perfect: True RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove ch2k_CO00MAL01_412, keep iso2k_1010. write decision to backup file 4280 4386 98 363 > 364/429,ch2k_QU96ESV01_422,iso2k_218,0.0,1.0 ==================================================================== === POTENTIAL DUPLICATE 363/429: ch2k_QU96ESV01_422+iso2k_218 === === URL 1: https://www.ncdc.noaa.gov/paleo/study/1839 === === URL 2: https://www.ncdc.noaa.gov/paleo/study/1839 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: True data_identical: False correlation_perfect: True RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove ch2k_QU96ESV01_422, keep iso2k_218. write decision to backup file 4281 4284 98 364 > 365/429,ch2k_DE13HAI01_424,ch2k_DE13HAI01_432,0.0,0.9999999999999998 ==================================================================== === POTENTIAL DUPLICATE 364/429: ch2k_DE13HAI01_424+ch2k_DE13HAI01_432 === === URL 1: https://www.ncdc.noaa.gov/paleo/study/34412 === === URL 2: https://www.ncdc.noaa.gov/paleo/study/34412 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: True data_identical: True correlation_perfect: True RECORDS IDENTICAL (identical data). Automatically choose #1. KEEP BLUE CIRCLES: keep ch2k_DE13HAI01_424, remove ch2k_DE13HAI01_432. write decision to backup file 4281 4696 98 365 > 366/429,ch2k_DE13HAI01_424,iso2k_1643,0.3336140951120854,0.9999999999999998 ==================================================================== === POTENTIAL DUPLICATE 365/429: ch2k_DE13HAI01_424+iso2k_1643 === === URL 1: https://www.ncdc.noaa.gov/paleo/study/34412 === === URL 2: https://agupubs.onlinelibrary.wiley.com/action/downloadSupplement?doi=10.1002%2F2016JC012458&file=jgrc22050-sup-0002-2016JC012458-ds01.xlsx === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: True RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove ch2k_DE13HAI01_424, keep iso2k_1643. write decision to backup file 4282 4283 98 366 > 367/429,ch2k_DE13HAI01_426,ch2k_DE13HAI01_430,0.0,1.0 ==================================================================== === POTENTIAL DUPLICATE 366/429: ch2k_DE13HAI01_426+ch2k_DE13HAI01_430 === === URL 1: https://www.ncdc.noaa.gov/paleo/study/34412 === === URL 2: https://www.ncdc.noaa.gov/paleo/study/34412 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: True data_identical: True correlation_perfect: True RECORDS IDENTICAL (identical data). Automatically choose #1. KEEP BLUE CIRCLES: keep ch2k_DE13HAI01_426, remove ch2k_DE13HAI01_430. write decision to backup file 4284 4696 98 367 > 368/429,ch2k_DE13HAI01_432,iso2k_1643,0.3336140951120854,0.9999999999999998 ==================================================================== === POTENTIAL DUPLICATE 367/429: ch2k_DE13HAI01_432+iso2k_1643 === === URL 1: https://www.ncdc.noaa.gov/paleo/study/34412 === === URL 2: https://agupubs.onlinelibrary.wiley.com/action/downloadSupplement?doi=10.1002%2F2016JC012458&file=jgrc22050-sup-0002-2016JC012458-ds01.xlsx === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: True RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove ch2k_DE13HAI01_432, keep iso2k_1643. write decision to backup file 4285 4592 98 368 > 369/429,ch2k_LI94SEC01_436,iso2k_1124,0.3335610731643405,1.0 ==================================================================== === POTENTIAL DUPLICATE 368/429: ch2k_LI94SEC01_436+iso2k_1124 === === URL 1: https://www.ncdc.noaa.gov/paleo/study/1853 === === URL 2: https://www.ncdc.noaa.gov/paleo/study/1853 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: True data_identical: False correlation_perfect: True RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove ch2k_LI94SEC01_436, keep iso2k_1124. write decision to backup file 4286 4287 98 369 > 370/429,ch2k_ZI15CLE01_438,ch2k_ZI15CLE01_440,0.0,0.9999999999999999 ==================================================================== === POTENTIAL DUPLICATE 369/429: ch2k_ZI15CLE01_438+ch2k_ZI15CLE01_440 === === URL 1: https://www.ncdc.noaa.gov/paleo/study/19239 === === URL 2: https://www.ncdc.noaa.gov/paleo/study/19239 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: True data_identical: True correlation_perfect: True RECORDS IDENTICAL (identical data). Automatically choose #1. KEEP BLUE CIRCLES: keep ch2k_ZI15CLE01_438, remove ch2k_ZI15CLE01_440. write decision to backup file 4290 4602 98 370 > 371/429,ch2k_TU01DEP01_450,iso2k_1201,0.0,1.0 ==================================================================== === POTENTIAL DUPLICATE 370/429: ch2k_TU01DEP01_450+iso2k_1201 === === URL 1: https://www.ncdc.noaa.gov/paleo/study/1866 === === URL 2: https://www.ncdc.noaa.gov/paleo/study/1866 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: True data_identical: False correlation_perfect: True RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove ch2k_TU01DEP01_450, keep iso2k_1201. write decision to backup file 4291 4460 98 371 > 372/429,ch2k_CO03PAL04_452,iso2k_513,0.0,1.0 ==================================================================== === POTENTIAL DUPLICATE 371/429: ch2k_CO03PAL04_452+iso2k_513 === === URL 1: https://www.ncdc.noaa.gov/paleo/study/1875 === === URL 2: https://www.ncdc.noaa.gov/paleo/study/1875 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: True data_identical: True correlation_perfect: True RECORDS IDENTICAL (identical data) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove ch2k_CO03PAL04_452, keep iso2k_513. write decision to backup file 4294 4324 98 372 > 373/429,ch2k_FL18DTO01_460,ch2k_FL18DTO02_554,0.34539124455148484,0.8307245439795362 ==================================================================== === POTENTIAL DUPLICATE 372/429: ch2k_FL18DTO01_460+ch2k_FL18DTO02_554 === === URL 1: https://www.ncdc.noaa.gov/paleo/study/34553 === === URL 2: https://www.ncdc.noaa.gov/paleo/study/34553 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False False (len(time_1)==len(time_2)) False metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: True data_identical: False correlation_perfect: False
**Decision required for this duplicate pair (see figure above).**
saved figure in /home/jupyter-lluecke/dod2k_v2.0/dod2k/figs//dup_detection/all_merged/372_ch2k_FL18DTO01_460_ch2k_FL18DTO02_554__4294_4324.jpg KEEP BOTH: keep ch2k_FL18DTO01_460, keep ch2k_FL18DTO02_554. write decision to backup file 4297 4298 98 373 > 374/429,ch2k_DU94URV01_468,ch2k_DU94URV01_470,0.0,1.0 ==================================================================== === POTENTIAL DUPLICATE 373/429: ch2k_DU94URV01_468+ch2k_DU94URV01_470 === === URL 1: https://www.ncdc.noaa.gov/paleo/study/1850 === === URL 2: https://www.ncdc.noaa.gov/paleo/study/1850 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: True data_identical: True correlation_perfect: True RECORDS IDENTICAL (identical data). Automatically choose #1. KEEP BLUE CIRCLES: keep ch2k_DU94URV01_468, remove ch2k_DU94URV01_470. write decision to backup file 4299 4465 98 374 > 375/429,ch2k_CO03PAL08_472,iso2k_523,0.0,1.0 ==================================================================== === POTENTIAL DUPLICATE 374/429: ch2k_CO03PAL08_472+iso2k_523 === === URL 1: https://www.ncdc.noaa.gov/paleo/study/1875 === === URL 2: https://www.ncdc.noaa.gov/paleo/study/1875 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: True data_identical: True correlation_perfect: True RECORDS IDENTICAL (identical data) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove ch2k_CO03PAL08_472, keep iso2k_523. write decision to backup file 4301 4302 98 375 > 376/429,ch2k_ZI14TUR01_480,ch2k_ZI14TUR01_482,0.0,1.0 ==================================================================== === POTENTIAL DUPLICATE 375/429: ch2k_ZI14TUR01_480+ch2k_ZI14TUR01_482 === === URL 1: https://www.ncdc.noaa.gov/paleo/study/16438 === === URL 2: https://www.ncdc.noaa.gov/paleo/study/16438 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: True data_identical: True correlation_perfect: True RECORDS IDENTICAL (identical data). Automatically choose #1. KEEP BLUE CIRCLES: keep ch2k_ZI14TUR01_480, remove ch2k_ZI14TUR01_482. write decision to backup file 4301 4412 98 376 > 377/429,ch2k_ZI14TUR01_480,iso2k_302,0.0,0.9999999994579302 ==================================================================== === POTENTIAL DUPLICATE 376/429: ch2k_ZI14TUR01_480+iso2k_302 === === URL 1: https://www.ncdc.noaa.gov/paleo/study/16438 === === URL 2: https://www.ncdc.noaa.gov/paleo/study/16438 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: True data_identical: False correlation_perfect: True RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove ch2k_ZI14TUR01_480, keep iso2k_302. write decision to backup file 4302 4412 98 377 > 378/429,ch2k_ZI14TUR01_482,iso2k_302,0.0,0.9999999994579302 ==================================================================== === POTENTIAL DUPLICATE 377/429: ch2k_ZI14TUR01_482+iso2k_302 === === URL 1: https://www.ncdc.noaa.gov/paleo/study/16438 === === URL 2: https://www.ncdc.noaa.gov/paleo/study/16438 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: True data_identical: False correlation_perfect: True RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove ch2k_ZI14TUR01_482, keep iso2k_302. write decision to backup file 4303 4679 98 378 > 379/429,ch2k_LI99CLI01_486,iso2k_1571,2.634776316918336,1.0 ==================================================================== === POTENTIAL DUPLICATE 378/429: ch2k_LI99CLI01_486+iso2k_1571 === === URL 1: https://www.ncdc.noaa.gov/paleo/study/1846 === === URL 2: https://www.ncdc.noaa.gov/paleo/study/1846 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: False lat True lon True elevation False archivetype True paleodata_proxy True sites_identical: False URL_identical: True data_identical: False correlation_perfect: True RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove ch2k_LI99CLI01_486, keep iso2k_1571. write decision to backup file 4304 4305 98 379 > 380/429,ch2k_ZI15BUN01_488,ch2k_ZI15BUN01_490,0.0,1.0 ==================================================================== === POTENTIAL DUPLICATE 379/429: ch2k_ZI15BUN01_488+ch2k_ZI15BUN01_490 === === URL 1: https://www.ncdc.noaa.gov/paleo/study/19239 === === URL 2: https://www.ncdc.noaa.gov/paleo/study/19239 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: True data_identical: True correlation_perfect: True RECORDS IDENTICAL (identical data). Automatically choose #1. KEEP BLUE CIRCLES: keep ch2k_ZI15BUN01_488, remove ch2k_ZI15BUN01_490. write decision to backup file 4306 4753 98 380 > 381/429,ch2k_FE18RUS01_492,iso2k_1861,1.0009991654937067,1.0 ==================================================================== === POTENTIAL DUPLICATE 380/429: ch2k_FE18RUS01_492+iso2k_1861 === === URL 1: https://doi.pangaea.de/10.1594/PANGAEA.891094 === === URL 2: https://www.ncdc.noaa.gov/paleo/study/1861 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: False lat True lon True elevation False archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: True RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove ch2k_FE18RUS01_492, keep iso2k_1861. write decision to backup file 4310 4311 98 381 > 382/429,ch2k_WU13TON01_504,ch2k_WU13TON01_506,0.0,1.0 ==================================================================== === POTENTIAL DUPLICATE 381/429: ch2k_WU13TON01_504+ch2k_WU13TON01_506 === === URL 1: https://www.ncdc.noaa.gov/paleo/study/15238 === === URL 2: https://www.ncdc.noaa.gov/paleo/study/15238 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: True data_identical: True correlation_perfect: True RECORDS IDENTICAL (identical data). Automatically choose #1. KEEP BLUE CIRCLES: keep ch2k_WU13TON01_504, remove ch2k_WU13TON01_506. write decision to backup file 4312 4315 98 382 > 383/429,ch2k_KI14PAR01_510,ch2k_KI14PAR01_518,0.0,0.9999999999999999 ==================================================================== === POTENTIAL DUPLICATE 382/429: ch2k_KI14PAR01_510+ch2k_KI14PAR01_518 === === URL 1: https://www.ncdc.noaa.gov/paleo/study/34953 === === URL 2: https://www.ncdc.noaa.gov/paleo/study/34953 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: True data_identical: True correlation_perfect: True RECORDS IDENTICAL (identical data). Automatically choose #1. KEEP BLUE CIRCLES: keep ch2k_KI14PAR01_510, remove ch2k_KI14PAR01_518. write decision to backup file 4313 4314 98 383 > 384/429,ch2k_KI14PAR01_512,ch2k_KI14PAR01_516,0.0,1.0 ==================================================================== === POTENTIAL DUPLICATE 383/429: ch2k_KI14PAR01_512+ch2k_KI14PAR01_516 === === URL 1: https://www.ncdc.noaa.gov/paleo/study/34953 === === URL 2: https://www.ncdc.noaa.gov/paleo/study/34953 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: True data_identical: True correlation_perfect: True RECORDS IDENTICAL (identical data). Automatically choose #1. KEEP BLUE CIRCLES: keep ch2k_KI14PAR01_512, remove ch2k_KI14PAR01_516. write decision to backup file 4316 4317 98 384 > 385/429,ch2k_ZI14IFR02_522,ch2k_ZI14IFR02_524,0.0,1.0 ==================================================================== === POTENTIAL DUPLICATE 384/429: ch2k_ZI14IFR02_522+ch2k_ZI14IFR02_524 === === URL 1: https://www.ncdc.noaa.gov/paleo/study/16438 === === URL 2: https://www.ncdc.noaa.gov/paleo/study/16438 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: True data_identical: True correlation_perfect: True RECORDS IDENTICAL (identical data). Automatically choose #1. KEEP BLUE CIRCLES: keep ch2k_ZI14IFR02_522, remove ch2k_ZI14IFR02_524. write decision to backup file 4325 4351 98 385 > 386/429,ch2k_BA04FIJ01_558,iso2k_55,0.5087587517215614,0.9999999999999999 ==================================================================== === POTENTIAL DUPLICATE 385/429: ch2k_BA04FIJ01_558+iso2k_55 === === URL 1: https://www.ncdc.noaa.gov/paleo/study/1881 === === URL 2: https://www.ncdc.noaa.gov/paleo/study/1881 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: True data_identical: False correlation_perfect: True RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove ch2k_BA04FIJ01_558, keep iso2k_55. write decision to backup file 4328 4423 98 386 > 387/429,ch2k_LI06FIJ01_582,iso2k_353,0.5087587517215614,1.0 ==================================================================== === POTENTIAL DUPLICATE 386/429: ch2k_LI06FIJ01_582+iso2k_353 === === URL 1: https://www.ncdc.noaa.gov/paleo/study/1003973 === === URL 2: https://www.ncdc.noaa.gov/paleo/study/16216 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: True RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #2. KEEP RED CROSSES: remove ch2k_LI06FIJ01_582, keep iso2k_353. write decision to backup file 4352 4581 98 387 > 388/429,iso2k_58,iso2k_1068,0.0,0.9997675035800241 ==================================================================== === POTENTIAL DUPLICATE 387/429: iso2k_58+iso2k_1068 === === URL 1: https://www.ncdc.noaa.gov/paleo/study/1881 === === URL 2: https://www.ncdc.noaa.gov/paleo/study/1916 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) False metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: False data_identical: False correlation_perfect: False
**Decision required for this duplicate pair (see figure above).** --------------------------------------------------------------------------------------------------------- ***ATTENTION*** THIS RECORD IS ASSOCIATED WITH MULTIPLE DUPLICATES! PLEASE PAY SPECIAL ATTENTION WHEN MAKING DECISIONS FOR THIS RECORD! The potential duplicates also associated with this record are: - pages2k_267 ---------------------------------------------------------------------------------------------------------
saved figure in /home/jupyter-lluecke/dod2k_v2.0/dod2k/figs//dup_detection/all_merged/387_iso2k_58_iso2k_1068__4352_4581.jpg KEEP RED CROSSES: remove iso2k_58, keep iso2k_1068. write decision to backup file 4362 4363 98 388 > 389/429,iso2k_94,iso2k_98,0.0,1.0 ==================================================================== === POTENTIAL DUPLICATE 388/429: iso2k_94+iso2k_98 === === URL 1: https://www.ncdc.noaa.gov/paleo/study/1859 === === URL 2: https://www.ncdc.noaa.gov/paleo/study/1859 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: True data_identical: True correlation_perfect: True RECORDS IDENTICAL (identical data). Automatically choose #1. KEEP BLUE CIRCLES: keep iso2k_94, remove iso2k_98. write decision to backup file 4370 4945 98 389 > 390/429,iso2k_120,sisal_253.0_171,0.06728323958312438,1.0 ==================================================================== === POTENTIAL DUPLICATE 389/429: iso2k_120+sisal_253.0_171 === === URL 1: https://www.ncdc.noaa.gov/paleo/study/6095 === === URL 2: ['10.1016/j.epsl.2007.10.015'] === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: False lat True lon True elevation True archivetype False paleodata_proxy True sites_identical: False URL_identical: False data_identical: True correlation_perfect: True RECORDS IDENTICAL (identical data) except for metadata. Automatically choose #1. KEEP BLUE CIRCLES: keep iso2k_120, remove sisal_253.0_171. write decision to backup file 4375 4958 98 390 > 391/429,iso2k_140,sisal_278.0_184,0.0,1.0 ==================================================================== === POTENTIAL DUPLICATE 390/429: iso2k_140+sisal_278.0_184 === === URL 1: https://www.ncdc.noaa.gov/paleo/study/8639 === === URL 2: ['10.1016/j.epsl.2009.12.017'] === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: False lat True lon True elevation True archivetype False paleodata_proxy True sites_identical: False URL_identical: False data_identical: True correlation_perfect: True RECORDS IDENTICAL (identical data) except for metadata. Automatically choose #1. KEEP BLUE CIRCLES: keep iso2k_140, remove sisal_278.0_184. write decision to backup file 4388 4915 98 391 > 392/429,iso2k_236,sisal_205.0_141,0.4690510096295847,1.0 ==================================================================== === POTENTIAL DUPLICATE 391/429: iso2k_236+sisal_205.0_141 === === URL 1: https://www.ncdc.noaa.gov/paleo/study/8632 === === URL 2: ['10.1038/ngeo605'] === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: False lat True lon True elevation False archivetype False paleodata_proxy True sites_identical: False URL_identical: False data_identical: True correlation_perfect: True RECORDS IDENTICAL (identical data) except for metadata. Automatically choose #1. KEEP BLUE CIRCLES: keep iso2k_236, remove sisal_205.0_141. write decision to backup file 4408 4409 98 392 > 393/429,iso2k_296,iso2k_298,0.0,1.0 ==================================================================== === POTENTIAL DUPLICATE 392/429: iso2k_296+iso2k_298 === === URL 1: https://www.ncdc.noaa.gov/paleo/study/22531 === === URL 2: https://www.ncdc.noaa.gov/paleo/study/22531 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: True data_identical: True correlation_perfect: True RECORDS IDENTICAL (identical data). Automatically choose #1. KEEP BLUE CIRCLES: keep iso2k_296, remove iso2k_298. write decision to backup file 4408 4410 98 393 > 394/429,iso2k_296,iso2k_299,0.0,1.0 ==================================================================== === POTENTIAL DUPLICATE 393/429: iso2k_296+iso2k_299 === === URL 1: https://www.ncdc.noaa.gov/paleo/study/22531 === === URL 2: https://www.ncdc.noaa.gov/paleo/study/22531 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: True data_identical: True correlation_perfect: True RECORDS IDENTICAL (identical data). Automatically choose #1. KEEP BLUE CIRCLES: keep iso2k_296, remove iso2k_299. write decision to backup file 4409 4410 98 394 > 395/429,iso2k_298,iso2k_299,0.0,1.0 ==================================================================== === POTENTIAL DUPLICATE 394/429: iso2k_298+iso2k_299 === === URL 1: https://www.ncdc.noaa.gov/paleo/study/22531 === === URL 2: https://www.ncdc.noaa.gov/paleo/study/22531 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: True data_identical: True correlation_perfect: True RECORDS IDENTICAL (identical data). Automatically choose #1. KEEP BLUE CIRCLES: keep iso2k_298, remove iso2k_299. write decision to backup file 4428 5066 98 395 > 396/429,iso2k_380,sisal_446.0_292,0.4944092732924204,1.0 ==================================================================== === POTENTIAL DUPLICATE 395/429: iso2k_380+sisal_446.0_292 === === URL 1: https://www.ncdc.noaa.gov/paleo-search/study/5441 === === URL 2: ['10.1016/j.epsl.2005.01.036'] === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: False lat True lon True elevation True archivetype False paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: True RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #1. KEEP BLUE CIRCLES: keep iso2k_380, remove sisal_446.0_292. write decision to backup file 4430 4521 98 396 > 397/429,iso2k_399,iso2k_806,0.0,1.0 ==================================================================== === POTENTIAL DUPLICATE 396/429: iso2k_399+iso2k_806 === === URL 1: https://www.ncdc.noaa.gov/paleo/study/21250 === === URL 2: https://doi.org/10.25921/6e73-as97 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: False data_identical: True correlation_perfect: True RECORDS IDENTICAL (identical data) except for metadata. Automatically choose #1. KEEP BLUE CIRCLES: keep iso2k_399, remove iso2k_806. write decision to backup file 4430 4522 98 397 > 398/429,iso2k_399,iso2k_811,0.0,1.0 ==================================================================== === POTENTIAL DUPLICATE 397/429: iso2k_399+iso2k_811 === === URL 1: https://www.ncdc.noaa.gov/paleo/study/21250 === === URL 2: https://doi.org/10.25921/6e73-as97 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: False data_identical: True correlation_perfect: True RECORDS IDENTICAL (identical data) except for metadata. Automatically choose #1. KEEP BLUE CIRCLES: keep iso2k_399, remove iso2k_811. write decision to backup file 4456 4482 98 398 > 399/429,iso2k_505,iso2k_579,0.0,0.9976024758754876 ==================================================================== === POTENTIAL DUPLICATE 398/429: iso2k_505+iso2k_579 === === URL 1: https://www.ncdc.noaa.gov/paleo/study/1875 === === URL 2: https://www.ncdc.noaa.gov/paleo/study/10373 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) False metadata_identical: False lat True lon True elevation False archivetype True paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: False
**Decision required for this duplicate pair (see figure above).**
---------------------------------------------------------------------------------------------------------
***ATTENTION*** THIS RECORD IS ASSOCIATED WITH MULTIPLE DUPLICATES! PLEASE PAY SPECIAL ATTENTION WHEN MAKING DECISIONS FOR THIS RECORD!
The potential duplicates also associated with this record are:
- pages2k_1488
............................................................
- Dataset ID : pages2k_1628
- URL : https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-Palmyra.Cobb.2003.txt
............................................................
- Dataset ID : ch2k_NU11PAL01_52
- URL : https://www.ncdc.noaa.gov/paleo/study/10373
---------------------------------------------------------------------------------------------------------
saved figure in /home/jupyter-lluecke/dod2k_v2.0/dod2k/figs//dup_detection/all_merged/398_iso2k_505_iso2k_579__4456_4482.jpg REMOVE BOTH: remove iso2k_579, remove iso2k_505. write decision to backup file 4468 4843 98 399 > 400/429,iso2k_533,sisal_115.0_69,0.4944092732924204,0.9999999999999998 ==================================================================== === POTENTIAL DUPLICATE 399/429: iso2k_533+sisal_115.0_69 === === URL 1: https://www.ncdc.noaa.gov/paleo-search/study/5427 === === URL 2: ['10.1126/science.1091220'] === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) False metadata_identical: False lat True lon True elevation True archivetype False paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: False
**Decision required for this duplicate pair (see figure above).**
saved figure in /home/jupyter-lluecke/dod2k_v2.0/dod2k/figs//dup_detection/all_merged/399_iso2k_533_sisal_115.0_69__4468_4843.jpg KEEP BOTH: keep iso2k_533, keep sisal_115.0_69. write decision to backup file 4471 4473 98 400 > 401/429,iso2k_546,iso2k_549,0.0,0.9992915900641419 ==================================================================== === POTENTIAL DUPLICATE 400/429: iso2k_546+iso2k_549 === === URL 1: this compilation === === URL 2: this compilation === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: True data_identical: False correlation_perfect: True RECORDS IDENTICAL (perfect correlation). Automatically choose #1. KEEP BLUE CIRCLES: keep iso2k_546, remove iso2k_549. write decision to backup file 4472 4474 98 401 > 402/429,iso2k_547,iso2k_550,0.0,0.9996157576286442 ==================================================================== === POTENTIAL DUPLICATE 401/429: iso2k_547+iso2k_550 === === URL 1: this compilation === === URL 2: this compilation === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: True data_identical: False correlation_perfect: True RECORDS IDENTICAL (perfect correlation). Automatically choose #1. KEEP BLUE CIRCLES: keep iso2k_547, remove iso2k_550. write decision to backup file 4501 4502 98 402 > 403/429,iso2k_702,iso2k_705,0.0,0.9755996990643611 ==================================================================== === POTENTIAL DUPLICATE 402/429: iso2k_702+iso2k_705 === === URL 1: https://www.ncdc.noaa.gov/paleo/study/1842 === === URL 2: https://www.ncdc.noaa.gov/paleo/study/1842 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False False (len(time_1)==len(time_2)) False metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: True data_identical: False correlation_perfect: False
**Decision required for this duplicate pair (see figure above).** --------------------------------------------------------------------------------------------------------- ***ATTENTION*** THIS RECORD IS ASSOCIATED WITH MULTIPLE DUPLICATES! PLEASE PAY SPECIAL ATTENTION WHEN MAKING DECISIONS FOR THIS RECORD! The potential duplicates also associated with this record are: - ch2k_GU99NAU01_314 ---------------------------------------------------------------------------------------------------------
saved figure in /home/jupyter-lluecke/dod2k_v2.0/dod2k/figs//dup_detection/all_merged/402_iso2k_702_iso2k_705__4501_4502.jpg KEEP RED CROSSES: remove iso2k_702, keep iso2k_705. write decision to backup file 4514 4515 98 403 > 404/429,iso2k_772,iso2k_775,0.0,1.0 ==================================================================== === POTENTIAL DUPLICATE 403/429: iso2k_772+iso2k_775 === === URL 1: https://www.ncdc.noaa.gov/paleo/study/9792 === === URL 2: https://www.ncdc.noaa.gov/paleo/study/9792 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: True data_identical: True correlation_perfect: True RECORDS IDENTICAL (identical data). Automatically choose #1. KEEP BLUE CIRCLES: keep iso2k_772, remove iso2k_775. write decision to backup file 4518 4519 98 404 > 405/429,iso2k_786,iso2k_788,0.0,0.9997021095883626 ==================================================================== === POTENTIAL DUPLICATE 404/429: iso2k_786+iso2k_788 === === URL 1: https://www.ncdc.noaa.gov/paleo/study/1856 === === URL 2: https://www.ncdc.noaa.gov/paleo/study/1856 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) False metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: True data_identical: False correlation_perfect: False
**Decision required for this duplicate pair (see figure above).** --------------------------------------------------------------------------------------------------------- ***ATTENTION*** THIS RECORD IS ASSOCIATED WITH MULTIPLE DUPLICATES! PLEASE PAY SPECIAL ATTENTION WHEN MAKING DECISIONS FOR THIS RECORD! The potential duplicates also associated with this record are: - ch2k_KU99HOU01_40 ---------------------------------------------------------------------------------------------------------
saved figure in /home/jupyter-lluecke/dod2k_v2.0/dod2k/figs//dup_detection/all_merged/404_iso2k_786_iso2k_788__4518_4519.jpg KEEP BLUE CIRCLES: keep iso2k_786, remove iso2k_788. write decision to backup file 4521 4522 98 405 > 406/429,iso2k_806,iso2k_811,0.0,1.0 ==================================================================== === POTENTIAL DUPLICATE 405/429: iso2k_806+iso2k_811 === === URL 1: https://doi.org/10.25921/6e73-as97 === === URL 2: https://doi.org/10.25921/6e73-as97 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: True data_identical: True correlation_perfect: True RECORDS IDENTICAL (identical data). Automatically choose #1. KEEP BLUE CIRCLES: keep iso2k_806, remove iso2k_811. write decision to backup file 4539 5088 98 406 > 407/429,iso2k_873,sisal_471.0_314,2.2239525755535694,0.9919284483744666 ==================================================================== === POTENTIAL DUPLICATE 406/429: iso2k_873+sisal_471.0_314 === === URL 1: https://www.ncdc.noaa.gov/paleo/study/8629 === === URL 2: ['10.1126/science.1163965'] === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: False lat True lon True elevation True archivetype False paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: True RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #1. KEEP BLUE CIRCLES: keep iso2k_873, remove sisal_471.0_314. write decision to backup file 4582 4701 98 407 > 408/429,iso2k_1069,iso2k_1660,0.0,0.9865701575318525 ==================================================================== === POTENTIAL DUPLICATE 407/429: iso2k_1069+iso2k_1660 === === URL 1: https://doi.pangaea.de/10.1594/PANGAEA.824732 === === URL 2: https://doi.pangaea.de/10.1594/PANGAEA.871279 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) False metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: False data_identical: False correlation_perfect: False
**Decision required for this duplicate pair (see figure above).** --------------------------------------------------------------------------------------------------------- ***ATTENTION*** THIS RECORD IS ASSOCIATED WITH MULTIPLE DUPLICATES! PLEASE PAY SPECIAL ATTENTION WHEN MAKING DECISIONS FOR THIS RECORD! The potential duplicates also associated with this record are: - pages2k_3599 ---------------------------------------------------------------------------------------------------------
saved figure in /home/jupyter-lluecke/dod2k_v2.0/dod2k/figs//dup_detection/all_merged/407_iso2k_1069_iso2k_1660__4582_4701.jpg KEEP BOTH: keep iso2k_1069, keep iso2k_1660. write decision to backup file 4588 4737 98 408 > 409/429,iso2k_1107,iso2k_1817,0.0,1.0 ==================================================================== === POTENTIAL DUPLICATE 408/429: iso2k_1107+iso2k_1817 === === URL 1: https://www.ncdc.noaa.gov/paleo/study/8640 === === URL 2: https://www.ncdc.noaa.gov/paleo-search/study/9741 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: False data_identical: True correlation_perfect: True RECORDS IDENTICAL (identical data) except for metadata. Automatically choose #1. KEEP BLUE CIRCLES: keep iso2k_1107, remove iso2k_1817. write decision to backup file 4588 4948 98 409 > 410/429,iso2k_1107,sisal_271.0_174,0.44453600976202207,1.0 ==================================================================== === POTENTIAL DUPLICATE 409/429: iso2k_1107+sisal_271.0_174 === === URL 1: https://www.ncdc.noaa.gov/paleo/study/8640 === === URL 2: ['10.1016/j.epsl.2010.04.002'] === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: False lat True lon True elevation True archivetype False paleodata_proxy True sites_identical: False URL_identical: False data_identical: True correlation_perfect: True RECORDS IDENTICAL (identical data) except for metadata. Automatically choose #1. KEEP BLUE CIRCLES: keep iso2k_1107, remove sisal_271.0_174. write decision to backup file 4599 4907 98 410 > 411/429,iso2k_1178,sisal_201.0_133,0.0,0.9999881881241324 ==================================================================== === POTENTIAL DUPLICATE 410/429: iso2k_1178+sisal_201.0_133 === === URL 1: https://www.ncdc.noaa.gov/paleo/study/12841 === === URL 2: ['10.1002/jqs.1490'] === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: False lat True lon True elevation True archivetype False paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: True RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #1. KEEP BLUE CIRCLES: keep iso2k_1178, remove sisal_201.0_133. write decision to backup file 4617 4618 98 411 > 412/429,iso2k_1283,iso2k_1286,0.0,1.0 ==================================================================== === POTENTIAL DUPLICATE 411/429: iso2k_1283+iso2k_1286 === === URL 1: https://www.ncdc.noaa.gov/paleo/study/17289 === === URL 2: https://www.ncdc.noaa.gov/paleo/study/17289 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: True data_identical: True correlation_perfect: True RECORDS IDENTICAL (identical data). Automatically choose #1. KEEP BLUE CIRCLES: keep iso2k_1283, remove iso2k_1286. write decision to backup file 4620 4987 98 412 > 413/429,iso2k_1288,sisal_329.0_213,0.36691187828569316,1.0 ==================================================================== === POTENTIAL DUPLICATE 412/429: iso2k_1288+sisal_329.0_213 === === URL 1: https://www.ncdc.noaa.gov/paleo-search/study/9742 === === URL 2: ['10.1016/j.epsl.2009.12.039'] === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: False lat True lon True elevation True archivetype False paleodata_proxy True sites_identical: False URL_identical: False data_identical: True correlation_perfect: True RECORDS IDENTICAL (identical data) except for metadata. Automatically choose #1. KEEP BLUE CIRCLES: keep iso2k_1288, remove sisal_329.0_213. write decision to backup file 4621 4989 98 413 > 414/429,iso2k_1291,sisal_330.0_215,0.36691187828569316,1.0 ==================================================================== === POTENTIAL DUPLICATE 413/429: iso2k_1291+sisal_330.0_215 === === URL 1: https://www.ncdc.noaa.gov/paleo-search/study/9742 === === URL 2: ['10.1016/j.epsl.2009.12.039'] === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: False lat True lon True elevation True archivetype False paleodata_proxy True sites_identical: False URL_identical: False data_identical: True correlation_perfect: True RECORDS IDENTICAL (identical data) except for metadata. Automatically choose #1. KEEP BLUE CIRCLES: keep iso2k_1291, remove sisal_330.0_215. write decision to backup file 4659 4973 98 414 > 415/429,iso2k_1495,sisal_305.0_199,0.09712205381798186,1.0 ==================================================================== === POTENTIAL DUPLICATE 414/429: iso2k_1495+sisal_305.0_199 === === URL 1: https://www.ncdc.noaa.gov/paleo/study/8637 === === URL 2: ['10.1029/2009gl040050'] === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: False lat True lon True elevation False archivetype False paleodata_proxy True sites_identical: False URL_identical: False data_identical: True correlation_perfect: True RECORDS IDENTICAL (identical data) except for metadata. Automatically choose #1. KEEP BLUE CIRCLES: keep iso2k_1495, remove sisal_305.0_199. write decision to backup file 4663 4840 98 415 > 416/429,iso2k_1504,sisal_113.0_66,1.3708902842296318,0.9999905278606608 ==================================================================== === POTENTIAL DUPLICATE 415/429: iso2k_1504+sisal_113.0_66 === === URL 1: https://www.ncdc.noaa.gov/paleo/study/13670 === === URL 2: ['10.1029/2012gl053936'] === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) False metadata_identical: False lat True lon True elevation False archivetype False paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: False
**Decision required for this duplicate pair (see figure above).**
saved figure in /home/jupyter-lluecke/dod2k_v2.0/dod2k/figs//dup_detection/all_merged/415_iso2k_1504_sisal_113.0_66__4663_4840.jpg KEEP BLUE CIRCLES: keep iso2k_1504, remove sisal_113.0_66. write decision to backup file 4672 4673 98 416 > 417/429,iso2k_1554,iso2k_1556,0.0,0.9995653924115724 ==================================================================== === POTENTIAL DUPLICATE 416/429: iso2k_1554+iso2k_1556 === === URL 1: https://www.ncdc.noaa.gov/paleo/study/1867 === === URL 2: https://www.ncdc.noaa.gov/paleo/study/1867 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) False metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: True data_identical: False correlation_perfect: False
**Decision required for this duplicate pair (see figure above).**
---------------------------------------------------------------------------------------------------------
***ATTENTION*** THIS RECORD IS ASSOCIATED WITH MULTIPLE DUPLICATES! PLEASE PAY SPECIAL ATTENTION WHEN MAKING DECISIONS FOR THIS RECORD!
The potential duplicates also associated with this record are:
- pages2k_3085
............................................................
- Dataset ID : ch2k_KU00NIN01_150
- URL : https://www.ncdc.noaa.gov/paleo/study/1867
---------------------------------------------------------------------------------------------------------
saved figure in /home/jupyter-lluecke/dod2k_v2.0/dod2k/figs//dup_detection/all_merged/416_iso2k_1554_iso2k_1556__4672_4673.jpg KEEP RED CROSSES: remove iso2k_1554, keep iso2k_1556. write decision to backup file 4705 4706 98 417 > 418/429,iso2k_1701,iso2k_1704,0.0,1.0 ==================================================================== === POTENTIAL DUPLICATE 417/429: iso2k_1701+iso2k_1704 === === URL 1: https://www.ncdc.noaa.gov/paleo/study/1891 === === URL 2: https://www.ncdc.noaa.gov/paleo/study/1891 === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: True data_identical: True correlation_perfect: True RECORDS IDENTICAL (identical data). Automatically choose #1. KEEP BLUE CIRCLES: keep iso2k_1701, remove iso2k_1704. write decision to backup file 4737 4948 98 418 > 419/429,iso2k_1817,sisal_271.0_174,0.44453600976202207,1.0 ==================================================================== === POTENTIAL DUPLICATE 418/429: iso2k_1817+sisal_271.0_174 === === URL 1: https://www.ncdc.noaa.gov/paleo-search/study/9741 === === URL 2: ['10.1016/j.epsl.2010.04.002'] === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: False lat True lon True elevation True archivetype False paleodata_proxy True sites_identical: False URL_identical: False data_identical: True correlation_perfect: True RECORDS IDENTICAL (identical data) except for metadata. Automatically choose #1. KEEP BLUE CIRCLES: keep iso2k_1817, remove sisal_271.0_174. write decision to backup file 4738 4951 98 419 > 420/429,iso2k_1820,sisal_272.0_177,0.44453600976202207,0.9999999999999999 ==================================================================== === POTENTIAL DUPLICATE 419/429: iso2k_1820+sisal_272.0_177 === === URL 1: https://www.ncdc.noaa.gov/paleo-search/study/9741 === === URL 2: ['10.1016/j.epsl.2010.04.002'] === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: False lat True lon True elevation True archivetype False paleodata_proxy True sites_identical: False URL_identical: False data_identical: True correlation_perfect: True RECORDS IDENTICAL (identical data) except for metadata. Automatically choose #1. KEEP BLUE CIRCLES: keep iso2k_1820, remove sisal_272.0_177. write decision to backup file 4739 4953 98 420 > 421/429,iso2k_1823,sisal_273.0_179,0.44453600976202207,0.9999999999999998 ==================================================================== === POTENTIAL DUPLICATE 420/429: iso2k_1823+sisal_273.0_179 === === URL 1: https://www.ncdc.noaa.gov/paleo-search/study/9741 === === URL 2: ['10.1016/j.epsl.2010.04.002'] === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: False lat True lon True elevation True archivetype False paleodata_proxy True sites_identical: False URL_identical: False data_identical: True correlation_perfect: True RECORDS IDENTICAL (identical data) except for metadata. Automatically choose #1. KEEP BLUE CIRCLES: keep iso2k_1823, remove sisal_273.0_179. write decision to backup file 4745 4751 98 421 > 422/429,iso2k_1848,iso2k_1855,0.0,1.0 ==================================================================== === POTENTIAL DUPLICATE 421/429: iso2k_1848+iso2k_1855 === === URL 1: http://www.iceandclimate.nbi.ku.dk/data/Vinther_etal_2010_data_02feb2010.xls === === URL 2: http://www.iceandclimate.nbi.ku.dk/data/Vinther_etal_2010_data_02feb2010.xls === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: True data_identical: True correlation_perfect: True RECORDS IDENTICAL (identical data). Automatically choose #1. KEEP BLUE CIRCLES: keep iso2k_1848, remove iso2k_1855. write decision to backup file 4746 4747 98 422 > 423/429,iso2k_1850,iso2k_1851,0.0,1.0 ==================================================================== === POTENTIAL DUPLICATE 422/429: iso2k_1850+iso2k_1851 === === URL 1: http://www.iceandclimate.nbi.ku.dk/data/Vinther_etal_2010_data_02feb2010.xls === === URL 2: http://www.iceandclimate.nbi.ku.dk/data/Vinther_etal_2010_data_02feb2010.xls === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: True data_identical: True correlation_perfect: True RECORDS IDENTICAL (identical data). Automatically choose #1. KEEP BLUE CIRCLES: keep iso2k_1850, remove iso2k_1851. write decision to backup file 4752 4968 98 423 > 424/429,iso2k_1856,sisal_294.0_194,0.011028565127834428,1.0 ==================================================================== === POTENTIAL DUPLICATE 423/429: iso2k_1856+sisal_294.0_194 === === URL 1: https://www.ncdc.noaa.gov/paleo/study/13543 === === URL 2: ['10.1038/ncomms2222'] === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: False lat True lon True elevation True archivetype False paleodata_proxy True sites_identical: False URL_identical: False data_identical: False correlation_perfect: True RECORDS IDENTICAL (perfect correlation) except for metadata. Automatically choose #1. KEEP BLUE CIRCLES: keep iso2k_1856, remove sisal_294.0_194. write decision to backup file 4792 4795 98 424 > 425/429,sisal_46.0_18,sisal_47.0_21,0.0,1.0 ==================================================================== === POTENTIAL DUPLICATE 424/429: sisal_46.0_18+sisal_47.0_21 === === URL 1: ['10.1191/095968399672625464'] === === URL 2: ['10.1029/2000gl012728'] === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: False data_identical: True correlation_perfect: True RECORDS IDENTICAL (identical data) except for metadata. Automatically choose #1. KEEP BLUE CIRCLES: keep sisal_46.0_18, remove sisal_47.0_21. write decision to backup file 4793 4796 98 425 > 426/429,sisal_46.0_19,sisal_47.0_22,0.0,1.0 ==================================================================== === POTENTIAL DUPLICATE 425/429: sisal_46.0_19+sisal_47.0_22 === === URL 1: ['10.1191/095968399672625464'] === === URL 2: ['10.1029/2000gl012728'] === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: False data_identical: True correlation_perfect: True RECORDS IDENTICAL (identical data) except for metadata. Automatically choose #1. KEEP BLUE CIRCLES: keep sisal_46.0_19, remove sisal_47.0_22. write decision to backup file 4794 4797 98 426 > 427/429,sisal_46.0_20,sisal_47.0_23,0.0,0.9999999999999999 ==================================================================== === POTENTIAL DUPLICATE 426/429: sisal_46.0_20+sisal_47.0_23 === === URL 1: ['10.1191/095968399672625464'] === === URL 2: ['10.1029/2000gl012728'] === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: False data_identical: True correlation_perfect: True RECORDS IDENTICAL (identical data) except for metadata. Automatically choose #1. KEEP BLUE CIRCLES: keep sisal_46.0_20, remove sisal_47.0_23. write decision to backup file 5044 5305 98 427 > 428/429,sisal_430.0_270,sisal_896.0_531,0.0,0.9999999999999999 ==================================================================== === POTENTIAL DUPLICATE 427/429: sisal_430.0_270+sisal_896.0_531 === === URL 1: ['10.1029/2017GL076838'] === === URL 2: ['10.1016/j.quascirev.2021.106822'] === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: False data_identical: True correlation_perfect: True RECORDS IDENTICAL (identical data) except for metadata. Automatically choose #1. KEEP BLUE CIRCLES: keep sisal_430.0_270, remove sisal_896.0_531. write decision to backup file 5045 5307 98 428 > 429/429,sisal_430.0_271,sisal_896.0_533,0.0,0.9999999999999999 ==================================================================== === POTENTIAL DUPLICATE 428/429: sisal_430.0_271+sisal_896.0_533 === === URL 1: ['10.1029/2017GL076838'] === === URL 2: ['10.1016/j.quascirev.2021.106822'] === True if pot_dup_corrs[i_pot_dups]>=0.98 else False True (len(time_1)==len(time_2)) True metadata_identical: True lat True lon True elevation True archivetype True paleodata_proxy True sites_identical: True URL_identical: False data_identical: True correlation_perfect: True RECORDS IDENTICAL (identical data) except for metadata. Automatically choose #1. KEEP BLUE CIRCLES: keep sisal_430.0_271, remove sisal_896.0_533. write decision to backup file ===================================================================== END OF DUPLICATE DECISION PROCESS. =====================================================================
(429, 27) Saved the decisions under data/all_merged/dup_detection/dup_decisions_all_merged_LL_25-12-11.csv Summary of all decisions made: #0: REMOVE record pages2k_0. KEEP record iso2k_296. #1: REMOVE record pages2k_0. KEEP record iso2k_298. #2: REMOVE record pages2k_0. KEEP record iso2k_299. #3: REMOVE record pages2k_6. KEEP record FE23_northamerica_usa_az555. #4: REMOVE record pages2k_50. KEEP record FE23_northamerica_canada_cana091. #5: KEEP record pages2k_62. KEEP record pages2k_63. #6: REMOVE record pages2k_81. KEEP record ch2k_HE08LRA01_76. #7: REMOVE record pages2k_81. KEEP record iso2k_1813. #8: REMOVE record pages2k_83. KEEP record iso2k_1916. #9: KEEP record pages2k_85. REMOVE record pages2k_88. #10: REMOVE record pages2k_94. KEEP record FE23_northamerica_canada_cana153. #11: REMOVE record pages2k_107. KEEP record FE23_northamerica_usa_ak046. #12: KEEP record pages2k_121. REMOVE record pages2k_122. #13: REMOVE record pages2k_132. KEEP record FE23_northamerica_canada_cana225. #14: REMOVE record pages2k_158. KEEP record FE23_northamerica_usa_wa069. #15: REMOVE record pages2k_171. KEEP record FE23_northamerica_usa_wy021. #16: REMOVE record pages2k_203. KEEP record iso2k_826. #17: REMOVE record pages2k_225. KEEP record FE23_northamerica_usa_nv512. #18: REMOVE record pages2k_238. KEEP record iso2k_1044. #19: REMOVE record pages2k_242. KEEP record ch2k_LI06FIJ01_582. #20: REMOVE record pages2k_242. KEEP record iso2k_353. #21: REMOVE record pages2k_258. KEEP record iso2k_1498. #22: REMOVE record pages2k_263. KEEP record iso2k_1322. #23: REMOVE record pages2k_267. KEEP record iso2k_58. #24: REMOVE record pages2k_267. KEEP record iso2k_1068. #25: REMOVE record pages2k_271. KEEP record ch2k_FE18RUS01_492. #26: REMOVE record pages2k_271. KEEP record iso2k_1861. #27: REMOVE record pages2k_273. KEEP record FE23_asia_russ130w. #28: REMOVE record pages2k_281. KEEP record FE23_northamerica_canada_cana155. #29: REMOVE record pages2k_294. KEEP record FE23_northamerica_usa_ak021. #30: KEEP record pages2k_305. REMOVE record pages2k_309. #31: REMOVE record pages2k_307. KEEP record pages2k_311. #32: REMOVE record pages2k_315. KEEP record iso2k_362. #33: REMOVE record pages2k_317. KEEP record ch2k_NA09MAL01_84. #34: REMOVE record pages2k_317. KEEP record iso2k_1754. #35: REMOVE record pages2k_323. KEEP record FE23_northamerica_canada_cana210. #36: REMOVE record pages2k_385. KEEP record ch2k_FE09OGA01_304. #37: REMOVE record pages2k_385. KEEP record iso2k_1922. #38: REMOVE record pages2k_387. KEEP record ch2k_FE09OGA01_306. #39: REMOVE record pages2k_395. KEEP record ch2k_CA07FLI01_400. #40: REMOVE record pages2k_395. KEEP record iso2k_1057. #41: REMOVE record pages2k_397. KEEP record ch2k_CA07FLI01_402. #42: REMOVE record pages2k_409. KEEP record ch2k_QU96ESV01_422. #43: REMOVE record pages2k_409. KEEP record iso2k_218. #44: KEEP record pages2k_414. REMOVE record pages2k_418. #45: KEEP record pages2k_417. REMOVE record pages2k_421. #46: KEEP record pages2k_427. REMOVE record pages2k_433. #47: KEEP record pages2k_435. REMOVE record pages2k_842. #48: KEEP record pages2k_444. REMOVE record pages2k_445. #49: KEEP record pages2k_444. REMOVE record pages2k_446. #50: KEEP record pages2k_445. REMOVE record pages2k_446. #51: REMOVE record pages2k_462. KEEP record ch2k_OS14UCP01_236. #52: REMOVE record pages2k_462. KEEP record iso2k_350. #53: KEEP record pages2k_468. REMOVE record pages2k_3550. #54: REMOVE record pages2k_468. KEEP record FE23_asia_russ137w. #55: KEEP record pages2k_472. REMOVE record pages2k_474. #56: KEEP record pages2k_472. KEEP record pages2k_477. #57: KEEP record pages2k_474. REMOVE record pages2k_477. #58: REMOVE record pages2k_478. KEEP record iso2k_1846. #59: REMOVE record pages2k_486. KEEP record FE23_northamerica_usa_ca609. #60: REMOVE record pages2k_495. KEEP record ch2k_LI06RAR01_12. #61: REMOVE record pages2k_495. KEEP record iso2k_1502. #62: REMOVE record pages2k_500. KEEP record ch2k_AS05GUA01_302. #63: REMOVE record pages2k_500. KEEP record iso2k_1559. #64: KEEP record pages2k_541. REMOVE record iso2k_404. #65: KEEP record pages2k_543. REMOVE record pages2k_976. #66: REMOVE record pages2k_565. KEEP record iso2k_998. #67: REMOVE record pages2k_583. KEEP record FE23_northamerica_usa_mt116. #68: REMOVE record pages2k_592. KEEP record ch2k_LI06RAR02_270. #69: REMOVE record pages2k_592. KEEP record iso2k_1500. #70: REMOVE record pages2k_610. KEEP record iso2k_1199. #71: REMOVE record pages2k_626. KEEP record FE23_northamerica_usa_wa071. #72: REMOVE record pages2k_691. KEEP record FE23_northamerica_canada_cana062. #73: REMOVE record pages2k_730. KEEP record iso2k_396. #74: REMOVE record pages2k_736. KEEP record FE23_northamerica_usa_wy024. #75: REMOVE record pages2k_800. KEEP record FE23_northamerica_canada_cana234. #76: REMOVE record pages2k_818. KEEP record iso2k_488. #77: KEEP record pages2k_827. REMOVE record pages2k_830. #78: KEEP record pages2k_831. REMOVE record pages2k_2220. #79: REMOVE record pages2k_831. KEEP record FE23_asia_russ127w. #80: REMOVE record pages2k_857. KEEP record FE23_northamerica_usa_ut511. #81: REMOVE record pages2k_881. KEEP record iso2k_1010. #82: KEEP record pages2k_893. REMOVE record pages2k_895. #83: KEEP record pages2k_893. REMOVE record pages2k_900. #84: KEEP record pages2k_895. REMOVE record pages2k_900. #85: REMOVE record pages2k_940. KEEP record ch2k_DR99ABR01_264. #86: REMOVE record pages2k_940. KEEP record ch2k_DR99ABR01_266. #87: REMOVE record pages2k_940. KEEP record iso2k_91. #88: REMOVE record pages2k_945. KEEP record iso2k_100. #89: REMOVE record pages2k_960. KEEP record iso2k_641. #90: REMOVE record pages2k_982. KEEP record FE23_northamerica_usa_or042. #91: REMOVE record pages2k_1004. KEEP record iso2k_644. #92: REMOVE record pages2k_1026. KEEP record FE23_northamerica_usa_az553. #93: REMOVE record pages2k_1048. KEEP record iso2k_1212. #94: REMOVE record pages2k_1089. KEEP record FE23_northamerica_usa_mt112. #95: REMOVE record pages2k_1089. KEEP record FE23_northamerica_usa_mt113. #96: REMOVE record pages2k_1108. KEEP record iso2k_1060. #97: REMOVE record pages2k_1116. KEEP record FE23_northamerica_canada_cana170w. #98: REMOVE record pages2k_1147. KEEP record ch2k_DA06MAF01_78. #99: REMOVE record pages2k_1147. KEEP record ch2k_DA06MAF02_104. #100: REMOVE record pages2k_1147. KEEP record iso2k_1748. #101: KEEP record pages2k_1153. REMOVE record pages2k_1156. #102: KEEP record pages2k_1153. REMOVE record pages2k_1160. #103: KEEP record pages2k_1156. REMOVE record pages2k_1160. #104: REMOVE record pages2k_1209. KEEP record FE23_northamerica_usa_co553. #105: REMOVE record pages2k_1252. KEEP record FE23_northamerica_canada_cana096. #106: REMOVE record pages2k_1274. KEEP record iso2k_1577. #107: REMOVE record pages2k_1293. KEEP record iso2k_821. #108: REMOVE record pages2k_1325. KEEP record FE23_northamerica_usa_wy030. #109: REMOVE record pages2k_1360. KEEP record ch2k_UR00MAI01_22. #110: REMOVE record pages2k_1360. KEEP record iso2k_94. #111: REMOVE record pages2k_1360. KEEP record iso2k_98. #112: KEEP record pages2k_1362. REMOVE record pages2k_1365. #113: REMOVE record pages2k_1370. KEEP record iso2k_1619. #114: REMOVE record pages2k_1420. KEEP record FE23_northamerica_canada_cana111. #115: KEEP record pages2k_1442. REMOVE record pages2k_1444. #116: REMOVE record pages2k_1488. KEEP record pages2k_1628. #117: REMOVE record pages2k_1488. KEEP record ch2k_NU11PAL01_52. #118: KEEP record pages2k_1488. REMOVE record iso2k_505. #119: REMOVE record pages2k_1488. KEEP record iso2k_579. #120: REMOVE record pages2k_1490. KEEP record ch2k_NU11PAL01_54. #121: REMOVE record pages2k_1491. KEEP record iso2k_575. #122: REMOVE record pages2k_1497. KEEP record iso2k_1885. #123: KEEP record pages2k_1515. REMOVE record pages2k_1519. #124: REMOVE record pages2k_1520. KEEP record pages2k_1522. #125: REMOVE record pages2k_1547. KEEP record iso2k_259. #126: REMOVE record pages2k_1566. KEEP record FE23_northamerica_canada_cana231. #127: REMOVE record pages2k_1605. KEEP record FE23_northamerica_usa_ca606. #128: REMOVE record pages2k_1619. KEEP record pages2k_1623. #129: KEEP record pages2k_1628. REMOVE record ch2k_NU11PAL01_52. #130: KEEP record pages2k_1628. REMOVE record iso2k_505. #131: KEEP record pages2k_1628. REMOVE record iso2k_579. #132: REMOVE record pages2k_1636. KEEP record FE23_northamerica_usa_wa081. #133: KEEP record pages2k_1686. KEEP record pages2k_1688. #134: REMOVE record pages2k_1692. KEEP record FE23_asia_mong012. #135: REMOVE record pages2k_1703. KEEP record ch2k_MO06PED01_226. #136: REMOVE record pages2k_1703. KEEP record iso2k_629. #137: REMOVE record pages2k_1712. KEEP record iso2k_715. #138: REMOVE record pages2k_1720. KEEP record iso2k_1579. #139: REMOVE record pages2k_1741. KEEP record FE23_northamerica_usa_wa104. #140: REMOVE record pages2k_1750. KEEP record iso2k_1856. #141: REMOVE record pages2k_1750. KEEP record sisal_294.0_194. #142: REMOVE record pages2k_1771. KEEP record ch2k_TU01LAI01_192. #143: REMOVE record pages2k_1804. KEEP record FE23_northamerica_usa_me010. #144: REMOVE record pages2k_1859. KEEP record ch2k_HE10GUA01_244. #145: REMOVE record pages2k_1859. KEEP record iso2k_1735. #146: REMOVE record pages2k_1861. KEEP record ch2k_HE10GUA01_246. #147: REMOVE record pages2k_1880. KEEP record FE23_northamerica_usa_ak060. #148: REMOVE record pages2k_1891. KEEP record pages2k_1893. #149: REMOVE record pages2k_1918. KEEP record iso2k_102. #150: KEEP record pages2k_1920. REMOVE record pages2k_1923. #151: REMOVE record pages2k_1932. KEEP record pages2k_1934. #152: REMOVE record pages2k_1942. KEEP record ch2k_ZI04IFR01_26. #153: REMOVE record pages2k_1942. KEEP record iso2k_257. #154: KEEP record pages2k_1972. REMOVE record pages2k_1973. #155: REMOVE record pages2k_1976. KEEP record pages2k_1980. #156: REMOVE record pages2k_1978. KEEP record pages2k_1983. #157: REMOVE record pages2k_1985. KEEP record iso2k_1294. #158: KEEP record pages2k_1989. REMOVE record pages2k_1991. #159: REMOVE record pages2k_1994. KEEP record ch2k_DE12ANC01_258. #160: REMOVE record pages2k_2013. KEEP record FE23_northamerica_canada_cana097. #161: REMOVE record pages2k_2042. KEEP record ch2k_TU95MAD01_24. #162: REMOVE record pages2k_2042. KEEP record iso2k_20. #163: REMOVE record pages2k_2059. KEEP record FE23_northamerica_usa_ak058. #164: REMOVE record pages2k_2085. KEEP record FE23_northamerica_canada_cana002. #165: REMOVE record pages2k_2094. KEEP record ch2k_TU01DEP01_450. #166: REMOVE record pages2k_2094. KEEP record iso2k_1201. #167: KEEP record pages2k_2098. KEEP record pages2k_2103. #168: REMOVE record pages2k_2110. KEEP record FE23_northamerica_usa_co554. #169: KEEP record pages2k_2146. KEEP record pages2k_2149. #170: KEEP record pages2k_2146. REMOVE record pages2k_2150. #171: KEEP record pages2k_2149. REMOVE record pages2k_2150. #172: REMOVE record pages2k_2156. KEEP record FE23_northamerica_canada_cana169w. #173: REMOVE record pages2k_2214. KEEP record iso2k_1631. #174: REMOVE record pages2k_2220. KEEP record FE23_asia_russ127w. #175: KEEP record pages2k_2226. REMOVE record FE23_asia_mong007w. #176: REMOVE record pages2k_2265. KEEP record FE23_northamerica_usa_ak070. #177: KEEP record pages2k_2287. REMOVE record pages2k_2290. #178: REMOVE record pages2k_2300. KEEP record ch2k_OS14RIP01_174. #179: KEEP record pages2k_2303. REMOVE record FE23_asia_mong006. #180: REMOVE record pages2k_2309. KEEP record ch2k_WE09ARR01_208. #181: REMOVE record pages2k_2311. KEEP record ch2k_WE09ARR01_210. #182: REMOVE record pages2k_2319. KEEP record FE23_northamerica_usa_ak6. #183: REMOVE record pages2k_2339. KEEP record pages2k_2344. #184: REMOVE record pages2k_2361. KEEP record FE23_northamerica_usa_wa097. #185: REMOVE record pages2k_2402. KEEP record FE23_northamerica_usa_co586. #186: REMOVE record pages2k_2430. KEEP record FE23_northamerica_canada_cana113. #187: REMOVE record pages2k_2473. KEEP record FE23_northamerica_usa_wy022. #188: REMOVE record pages2k_2500. KEEP record pages2k_2502. #189: REMOVE record pages2k_2510. KEEP record iso2k_1626. #190: REMOVE record pages2k_2514. KEEP record iso2k_1467. #191: REMOVE record pages2k_2517. KEEP record iso2k_1130. #192: REMOVE record pages2k_2534. KEEP record iso2k_1575. #193: REMOVE record pages2k_2538. KEEP record iso2k_1862. #194: REMOVE record pages2k_2561. KEEP record FE23_northamerica_canada_cana094. #195: REMOVE record pages2k_2592. KEEP record pages2k_2596. #196: REMOVE record pages2k_2595. KEEP record pages2k_2599. #197: KEEP record pages2k_2604. REMOVE record pages2k_2606. #198: REMOVE record pages2k_2604. KEEP record iso2k_1481. #199: REMOVE record pages2k_2606. KEEP record iso2k_1481. #200: KEEP record pages2k_2607. REMOVE record pages2k_2609. #201: KEEP record pages2k_2607. REMOVE record pages2k_2612. #202: KEEP record pages2k_2609. REMOVE record pages2k_2612. #203: REMOVE record pages2k_2613. KEEP record iso2k_1470. #204: REMOVE record pages2k_2617. KEEP record iso2k_1573. #205: REMOVE record pages2k_2634. KEEP record FE23_northamerica_usa_id013. #206: REMOVE record pages2k_2660. KEEP record FE23_northamerica_usa_ak014. #207: REMOVE record pages2k_2677. KEEP record FE23_northamerica_usa_wy023. #208: REMOVE record pages2k_2703. KEEP record FE23_northamerica_usa_ak094. #209: REMOVE record pages2k_2722. KEEP record FE23_northamerica_canada_cana238. #210: REMOVE record pages2k_2750. KEEP record iso2k_1708. #211: KEEP record pages2k_2752. REMOVE record pages2k_2755. #212: KEEP record pages2k_2752. REMOVE record pages2k_2759. #213: KEEP record pages2k_2755. REMOVE record pages2k_2759. #214: KEEP record pages2k_2793. REMOVE record pages2k_2795. #215: KEEP record pages2k_2795. KEEP record pages2k_2798. #216: KEEP record pages2k_2796. REMOVE record pages2k_2798. #217: REMOVE record pages2k_2830. KEEP record FE23_northamerica_mexico_mexi020. #218: REMOVE record pages2k_2843. KEEP record FE23_northamerica_usa_wa083. #219: REMOVE record pages2k_2899. KEEP record pages2k_2901. #220: KEEP record pages2k_2904. REMOVE record pages2k_2906. #221: REMOVE record pages2k_2922. KEEP record FE23_northamerica_usa_ca603. #222: REMOVE record pages2k_2953. KEEP record iso2k_573. #223: REMOVE record pages2k_2959. KEEP record FE23_northamerica_mexico_mexi043. #224: REMOVE record pages2k_2976. KEEP record FE23_northamerica_usa_id008. #225: REMOVE record pages2k_3002. KEEP record FE23_northamerica_usa_or043. #226: KEEP record pages2k_3028. REMOVE record pages2k_3030. #227: KEEP record pages2k_3028. REMOVE record pages2k_3033. #228: KEEP record pages2k_3030. REMOVE record pages2k_3033. #229: REMOVE record pages2k_3038. KEEP record FE23_northamerica_usa_mt108. #230: REMOVE record pages2k_3064. KEEP record iso2k_698. #231: REMOVE record pages2k_3068. KEEP record ch2k_ZI14IFR02_522. #232: REMOVE record pages2k_3068. KEEP record ch2k_ZI14IFR02_524. #233: REMOVE record pages2k_3085. KEEP record ch2k_KU00NIN01_150. #234: REMOVE record pages2k_3085. KEEP record iso2k_1554. #235: REMOVE record pages2k_3085. KEEP record iso2k_1556. #236: REMOVE record pages2k_3107. KEEP record FE23_northamerica_usa_co552. #237: REMOVE record pages2k_3108. KEEP record FE23_northamerica_usa_co552. #238: REMOVE record pages2k_3132. KEEP record ch2k_QU06RAB01_144. #239: REMOVE record pages2k_3132. KEEP record iso2k_1311. #240: REMOVE record pages2k_3134. KEEP record ch2k_QU06RAB01_146. #241: REMOVE record pages2k_3170. KEEP record FE23_australia_newz062. #242: REMOVE record pages2k_3179. KEEP record FE23_northamerica_usa_ak057. #243: KEEP record pages2k_3188. REMOVE record pages2k_3191. #244: KEEP record pages2k_3196. REMOVE record FE23_asia_mong011. #245: REMOVE record pages2k_3202. KEEP record iso2k_1727. #246: KEEP record pages2k_3234. REMOVE record pages2k_3236. #247: KEEP record pages2k_3234. REMOVE record pages2k_3239. #248: KEEP record pages2k_3236. REMOVE record pages2k_3239. #249: REMOVE record pages2k_3243. KEEP record iso2k_0. #250: REMOVE record pages2k_3263. KEEP record iso2k_1264. #251: REMOVE record pages2k_3266. KEEP record ch2k_GO12SBV01_396. #252: REMOVE record pages2k_3266. KEEP record iso2k_870. #253: REMOVE record pages2k_3307. KEEP record iso2k_339. #254: REMOVE record pages2k_3313. KEEP record FE23_northamerica_usa_ca560. #255: KEEP record pages2k_3337. REMOVE record pages2k_3342. #256: REMOVE record pages2k_3352. KEEP record ch2k_ZI14TUR01_480. #257: REMOVE record pages2k_3352. KEEP record ch2k_ZI14TUR01_482. #258: REMOVE record pages2k_3352. KEEP record iso2k_302. #259: REMOVE record pages2k_3372. KEEP record ch2k_KI04MCV01_366. #260: REMOVE record pages2k_3372. KEEP record iso2k_155. #261: REMOVE record pages2k_3374. KEEP record ch2k_KI04MCV01_368. #262: REMOVE record pages2k_3404. KEEP record FE23_northamerica_canada_cana029. #263: KEEP record pages2k_3417. REMOVE record pages2k_3419. #264: REMOVE record pages2k_3503. KEEP record FE23_northamerica_usa_wa072. #265: REMOVE record pages2k_3524. KEEP record FE23_northamerica_usa_ak010. #266: REMOVE record pages2k_3550. KEEP record FE23_asia_russ137w. #267: REMOVE record pages2k_3552. KEEP record iso2k_1581. #268: REMOVE record pages2k_3554. KEEP record ch2k_LI94SEC01_436. #269: REMOVE record pages2k_3554. KEEP record iso2k_1124. #270: REMOVE record pages2k_3571. KEEP record iso2k_174. #271: REMOVE record pages2k_3583. KEEP record FE23_northamerica_usa_co633. #272: REMOVE record pages2k_3599. KEEP record iso2k_1069. #273: REMOVE record pages2k_3599. KEEP record iso2k_1660. #274: REMOVE record pages2k_3609. KEEP record FE23_northamerica_canada_cana053. #275: REMOVE record pages2k_3631. KEEP record iso2k_1530. #276: REMOVE record pages2k_3642. KEEP record FE23_northamerica_usa_wy025. #277: KEEP record FE23_southamerica_arge016. REMOVE record FE23_southamerica_arge085. #278: REMOVE record FE23_northamerica_canada_cana100. KEEP record FE23_northamerica_canada_cana213. #279: REMOVE record FE23_northamerica_canada_cana105. KEEP record FE23_northamerica_canada_cana217. #280: KEEP record FE23_northamerica_canada_cana116. REMOVE record FE23_northamerica_canada_cana168w. #281: KEEP record FE23_northamerica_canada_cana161. REMOVE record FE23_northamerica_canada_cana162. #282: KEEP record FE23_southamerica_chil016. REMOVE record FE23_southamerica_chil017. #283: KEEP record FE23_europe_swed019w. REMOVE record FE23_europe_swed021w. #284: REMOVE record FE23_northamerica_mexico_mexi022. KEEP record FE23_northamerica_mexico_mexi023. #285: REMOVE record FE23_australia_newz003. KEEP record FE23_australia_newz060. #286: KEEP record FE23_australia_newz008. REMOVE record FE23_australia_newz092. #287: REMOVE record FE23_australia_newz014. KEEP record FE23_australia_newz061. #288: REMOVE record FE23_australia_newz018. KEEP record FE23_australia_newz062. #289: REMOVE record FE23_australia_newz019. KEEP record FE23_australia_newz063. #290: REMOVE record FE23_northamerica_usa_ca066. KEEP record FE23_northamerica_usa_ca628. #291: REMOVE record FE23_northamerica_usa_ca067. KEEP record FE23_northamerica_usa_ca628. #292: REMOVE record FE23_northamerica_usa_ca512. KEEP record FE23_northamerica_usa_ca613. #293: REMOVE record FE23_northamerica_usa_ca535. KEEP record FE23_northamerica_usa_ca670. #294: COMPOSITE record FE23_northamerica_usa_me017. COMPOSITE record FE23_northamerica_usa_me018. #295: KEEP record FE23_northamerica_usa_mo. REMOVE record FE23_northamerica_usa_mo009. #296: COMPOSITE record FE23_northamerica_usa_mt112. COMPOSITE record FE23_northamerica_usa_mt113. #297: COMPOSITE record FE23_northamerica_usa_nj001. COMPOSITE record FE23_northamerica_usa_nj002. #298: KEEP record FE23_northamerica_usa_nm024. REMOVE record FE23_northamerica_usa_nm055. #299: REMOVE record FE23_northamerica_usa_nv060. KEEP record FE23_northamerica_usa_nv518. #300: REMOVE record FE23_northamerica_usa_nv512. KEEP record FE23_northamerica_usa_nv521. #301: REMOVE record FE23_northamerica_usa_nv513. KEEP record FE23_northamerica_usa_nv520. #302: KEEP record ch2k_ZI15MER01_2. REMOVE record ch2k_ZI15MER01_4. #303: REMOVE record ch2k_CO03PAL03_6. KEEP record iso2k_511. #304: REMOVE record ch2k_CO03PAL02_8. KEEP record iso2k_509. #305: REMOVE record ch2k_LI06RAR01_12. KEEP record iso2k_1502. #306: REMOVE record ch2k_CO03PAL07_14. KEEP record iso2k_521. #307: REMOVE record ch2k_UR00MAI01_22. KEEP record iso2k_94. #308: REMOVE record ch2k_UR00MAI01_22. KEEP record iso2k_98. #309: REMOVE record ch2k_TU95MAD01_24. KEEP record iso2k_20. #310: REMOVE record ch2k_ZI04IFR01_26. KEEP record iso2k_257. #311: REMOVE record ch2k_RE18CAY01_30. KEEP record iso2k_917. #312: REMOVE record ch2k_KU99HOU01_40. KEEP record iso2k_786. #313: KEEP record ch2k_KU99HOU01_40. REMOVE record iso2k_788. #314: KEEP record ch2k_NU11PAL01_52. KEEP record iso2k_505. #315: REMOVE record ch2k_NU11PAL01_52. KEEP record iso2k_579. #316: REMOVE record ch2k_CA14TIM01_64. KEEP record iso2k_473. #317: REMOVE record ch2k_HE08LRA01_76. KEEP record iso2k_1813. #318: REMOVE record ch2k_DA06MAF01_78. KEEP record iso2k_1748. #319: REMOVE record ch2k_NA09MAL01_84. KEEP record iso2k_1754. #320: REMOVE record ch2k_SW98STP01_86. KEEP record iso2k_50. #321: REMOVE record ch2k_DA06MAF02_104. KEEP record iso2k_1748. #322: REMOVE record ch2k_CO03PAL01_110. KEEP record iso2k_507. #323: REMOVE record ch2k_CH98PIR01_116. KEEP record iso2k_1229. #324: KEEP record ch2k_XI17HAI01_128. REMOVE record ch2k_XI17HAI01_136. #325: REMOVE record ch2k_XI17HAI01_128. KEEP record iso2k_1762. #326: KEEP record ch2k_XI17HAI01_130. REMOVE record ch2k_XI17HAI01_134. #327: REMOVE record ch2k_XI17HAI01_136. KEEP record iso2k_1762. #328: KEEP record ch2k_DE14DTO03_140. KEEP record ch2k_DE14DTO01_148. #329: REMOVE record ch2k_QU06RAB01_144. KEEP record iso2k_1311. #330: REMOVE record ch2k_KU00NIN01_150. KEEP record iso2k_1554. #331: KEEP record ch2k_KU00NIN01_150. REMOVE record iso2k_1556. #332: KEEP record ch2k_EV18ROC01_184. REMOVE record ch2k_EV18ROC01_186. #333: REMOVE record ch2k_CA13SAP01_188. KEEP record iso2k_569. #334: KEEP record ch2k_HE13MIS01_194. KEEP record iso2k_211. #335: KEEP record ch2k_HE13MIS01_194. REMOVE record iso2k_213. #336: KEEP record ch2k_ZI15IMP02_200. REMOVE record ch2k_ZI15IMP02_202. #337: REMOVE record ch2k_PF04PBA01_204. KEEP record iso2k_1701. #338: REMOVE record ch2k_PF04PBA01_204. KEEP record iso2k_1704. #339: REMOVE record ch2k_CO03PAL05_212. KEEP record iso2k_515. #340: REMOVE record ch2k_MO06PED01_226. KEEP record iso2k_629. #341: REMOVE record ch2k_OS14UCP01_236. KEEP record iso2k_350. #342: REMOVE record ch2k_HE10GUA01_244. KEEP record iso2k_1735. #343: KEEP record ch2k_DR99ABR01_264. REMOVE record ch2k_DR99ABR01_266. #344: REMOVE record ch2k_DR99ABR01_264. KEEP record iso2k_91. #345: REMOVE record ch2k_DR99ABR01_266. KEEP record iso2k_91. #346: REMOVE record ch2k_LI06RAR02_270. KEEP record iso2k_1500. #347: KEEP record ch2k_ZI15TAN01_278. REMOVE record ch2k_ZI15TAN01_280. #348: REMOVE record ch2k_AS05GUA01_302. KEEP record iso2k_1559. #349: REMOVE record ch2k_FE09OGA01_304. KEEP record iso2k_1922. #350: KEEP record ch2k_GU99NAU01_314. REMOVE record iso2k_702. #351: REMOVE record ch2k_GU99NAU01_314. KEEP record iso2k_705. #352: REMOVE record ch2k_CO03PAL10_324. KEEP record iso2k_519. #353: KEEP record ch2k_ZI15IMP01_328. REMOVE record ch2k_ZI15IMP01_330. #354: KEEP record ch2k_RO19YUC01_338. REMOVE record ch2k_RO19YUC01_340. #355: REMOVE record ch2k_CO03PAL09_358. KEEP record iso2k_525. #356: REMOVE record ch2k_KI04MCV01_366. KEEP record iso2k_155. #357: REMOVE record ch2k_BA04FIJ02_382. KEEP record iso2k_52. #358: REMOVE record ch2k_CO03PAL06_386. KEEP record iso2k_517. #359: REMOVE record ch2k_GO12SBV01_396. KEEP record iso2k_870. #360: REMOVE record ch2k_CA07FLI01_400. KEEP record iso2k_1057. #361: REMOVE record ch2k_CO93TAR01_408. KEEP record iso2k_539. #362: REMOVE record ch2k_CO00MAL01_412. KEEP record iso2k_1010. #363: REMOVE record ch2k_QU96ESV01_422. KEEP record iso2k_218. #364: KEEP record ch2k_DE13HAI01_424. REMOVE record ch2k_DE13HAI01_432. #365: REMOVE record ch2k_DE13HAI01_424. KEEP record iso2k_1643. #366: KEEP record ch2k_DE13HAI01_426. REMOVE record ch2k_DE13HAI01_430. #367: REMOVE record ch2k_DE13HAI01_432. KEEP record iso2k_1643. #368: REMOVE record ch2k_LI94SEC01_436. KEEP record iso2k_1124. #369: KEEP record ch2k_ZI15CLE01_438. REMOVE record ch2k_ZI15CLE01_440. #370: REMOVE record ch2k_TU01DEP01_450. KEEP record iso2k_1201. #371: REMOVE record ch2k_CO03PAL04_452. KEEP record iso2k_513. #372: KEEP record ch2k_FL18DTO01_460. KEEP record ch2k_FL18DTO02_554. #373: KEEP record ch2k_DU94URV01_468. REMOVE record ch2k_DU94URV01_470. #374: REMOVE record ch2k_CO03PAL08_472. KEEP record iso2k_523. #375: KEEP record ch2k_ZI14TUR01_480. REMOVE record ch2k_ZI14TUR01_482. #376: REMOVE record ch2k_ZI14TUR01_480. KEEP record iso2k_302. #377: REMOVE record ch2k_ZI14TUR01_482. KEEP record iso2k_302. #378: REMOVE record ch2k_LI99CLI01_486. KEEP record iso2k_1571. #379: KEEP record ch2k_ZI15BUN01_488. REMOVE record ch2k_ZI15BUN01_490. #380: REMOVE record ch2k_FE18RUS01_492. KEEP record iso2k_1861. #381: KEEP record ch2k_WU13TON01_504. REMOVE record ch2k_WU13TON01_506. #382: KEEP record ch2k_KI14PAR01_510. REMOVE record ch2k_KI14PAR01_518. #383: KEEP record ch2k_KI14PAR01_512. REMOVE record ch2k_KI14PAR01_516. #384: KEEP record ch2k_ZI14IFR02_522. REMOVE record ch2k_ZI14IFR02_524. #385: REMOVE record ch2k_BA04FIJ01_558. KEEP record iso2k_55. #386: REMOVE record ch2k_LI06FIJ01_582. KEEP record iso2k_353. #387: REMOVE record iso2k_58. KEEP record iso2k_1068. #388: KEEP record iso2k_94. REMOVE record iso2k_98. #389: KEEP record iso2k_120. REMOVE record sisal_253.0_171. #390: KEEP record iso2k_140. REMOVE record sisal_278.0_184. #391: KEEP record iso2k_236. REMOVE record sisal_205.0_141. #392: KEEP record iso2k_296. REMOVE record iso2k_298. #393: KEEP record iso2k_296. REMOVE record iso2k_299. #394: KEEP record iso2k_298. REMOVE record iso2k_299. #395: KEEP record iso2k_380. REMOVE record sisal_446.0_292. #396: KEEP record iso2k_399. REMOVE record iso2k_806. #397: KEEP record iso2k_399. REMOVE record iso2k_811. #398: REMOVE record iso2k_505. REMOVE record iso2k_579. #399: KEEP record iso2k_533. KEEP record sisal_115.0_69. #400: KEEP record iso2k_546. REMOVE record iso2k_549. #401: KEEP record iso2k_547. REMOVE record iso2k_550. #402: REMOVE record iso2k_702. KEEP record iso2k_705. #403: KEEP record iso2k_772. REMOVE record iso2k_775. #404: KEEP record iso2k_786. REMOVE record iso2k_788. #405: KEEP record iso2k_806. REMOVE record iso2k_811. #406: KEEP record iso2k_873. REMOVE record sisal_471.0_314. #407: KEEP record iso2k_1069. KEEP record iso2k_1660. #408: KEEP record iso2k_1107. REMOVE record iso2k_1817. #409: KEEP record iso2k_1107. REMOVE record sisal_271.0_174. #410: KEEP record iso2k_1178. REMOVE record sisal_201.0_133. #411: KEEP record iso2k_1283. REMOVE record iso2k_1286. #412: KEEP record iso2k_1288. REMOVE record sisal_329.0_213. #413: KEEP record iso2k_1291. REMOVE record sisal_330.0_215. #414: KEEP record iso2k_1495. REMOVE record sisal_305.0_199. #415: KEEP record iso2k_1504. REMOVE record sisal_113.0_66. #416: REMOVE record iso2k_1554. KEEP record iso2k_1556. #417: KEEP record iso2k_1701. REMOVE record iso2k_1704. #418: KEEP record iso2k_1817. REMOVE record sisal_271.0_174. #419: KEEP record iso2k_1820. REMOVE record sisal_272.0_177. #420: KEEP record iso2k_1823. REMOVE record sisal_273.0_179. #421: KEEP record iso2k_1848. REMOVE record iso2k_1855. #422: KEEP record iso2k_1850. REMOVE record iso2k_1851. #423: KEEP record iso2k_1856. REMOVE record sisal_294.0_194. #424: KEEP record sisal_46.0_18. REMOVE record sisal_47.0_21. #425: KEEP record sisal_46.0_19. REMOVE record sisal_47.0_22. #426: KEEP record sisal_46.0_20. REMOVE record sisal_47.0_23. #427: KEEP record sisal_430.0_270. REMOVE record sisal_896.0_531. #428: KEEP record sisal_430.0_271. REMOVE record sisal_896.0_533.
In [31]:
Copied!
date = str(datetime.datetime.utcnow())[2:10]
fn = utf.find(f'dup_decisions_{df.name}_{initials}_{date}.csv', f'data/{df.name}/dup_detection')
if fn != []:
print('----------------------------------------------------')
print('Successfully finished the duplicate decision process!'.upper())
print('----------------------------------------------------')
print('Saved the decision output file in:')
print()
for ff in fn:
print('%s.'%ff)
print()
print('You are now able to proceed with the next notebook: dup_removal.ipynb')
else:
print('!!!!!!!!!!!!WARNING!!!!!!!!!!!')
print('Final output file is missing.')
print()
print('Please re-run the notebook to complete duplicate decision process.')
date = str(datetime.datetime.utcnow())[2:10]
fn = utf.find(f'dup_decisions_{df.name}_{initials}_{date}.csv', f'data/{df.name}/dup_detection')
if fn != []:
print('----------------------------------------------------')
print('Successfully finished the duplicate decision process!'.upper())
print('----------------------------------------------------')
print('Saved the decision output file in:')
print()
for ff in fn:
print('%s.'%ff)
print()
print('You are now able to proceed with the next notebook: dup_removal.ipynb')
else:
print('!!!!!!!!!!!!WARNING!!!!!!!!!!!')
print('Final output file is missing.')
print()
print('Please re-run the notebook to complete duplicate decision process.')
---------------------------------------------------- SUCCESSFULLY FINISHED THE DUPLICATE DECISION PROCESS! ---------------------------------------------------- Saved the decision output file in: data/all_merged/dup_detection/dup_decisions_all_merged_LL_25-12-11.csv. You are now able to proceed with the next notebook: dup_removal.ipynb
In [ ]:
Copied!